This project is read-only.

Include large blobs in sync to SQL Server

Jun 28, 2013 at 8:37 AM
Hi, Selvin,

I saw that the recent code version includes some changes to add blob columns to the sync.
Can you please:
  • confirm that this is working
  • indicate what it the largest blob that can be sync'ed - what is the largest blob that you plan to sync?
The camera on my tablet is 8Mb and the Galaxy S4 phone is now 14MB. I think it is natural that users will need to sync such large photos, and I need to sync up these large photos now.

Thanks for you assistance,
Geoff
Jun 28, 2013 at 8:49 AM
this is my current production code ... and my production app needs blob ... so, yes it is working.
you need to add:
  <system.web>
    <httpRuntime  maxRequestLength="102400" executionTimeout="1000"/>
    <!-- you can play  play with numbers-->
  </system.web>
and
<system.serviceModel>
    <webHttpBinding>
       <binding name="" maxReceivedMessageSize="104857600"/>
        <!-- you can play with numbers-->
    </webHttpBinding>
</system.serviceModel>
to your web.config

in my production code i'm resizing images taken from camera since i don't need 'em at full size and everything is working as it should

remeber that every change in the row with blob will send whole row
Jun 28, 2013 at 9:08 AM
Hi, Selvin,

Wow, thanks for the fast response!
I may have the option of scaling the image down to 1/2 dimension or so, or even a fixed size - will have to review with the client.
I will implement the code for a trial as soon as possible - 2 or 3 days since its now weekend for me.

In regard to the image row syncing every time, once the sync data is processed, I will be transferring the image to another table, then clearing the field on the SQL side, so it should then delete from the SQLite table...

Regards,
Geoff
Jul 12, 2013 at 9:32 AM
Hi, Selvin,

I am trying to configure the 'message size' section in web.config for an IIS6 (Windows Server2003) service.
I can sync small images (26kB), so no changes required, but not larger ones (changes are required).

Would you mind posting the full xml blocks for the sections system.servicemodel block of web.config?
I can't find much config info for the full section on the net, and my (SyncFx based) web.config never had such a section to use as a basis.

Thanks,
Geoff
Jul 12, 2013 at 9:36 AM
Jul 15, 2013 at 10:57 AM
Edited Jul 15, 2013 at 11:00 AM
Hi, Selvin,

I have implemented the web.config settings above, as well as the equivalent in IIS Metabase.
I have spent a lot of time trying to identify the issue, but with no success.
I can sync only smaller images/blobs - say up to 46kb (probably 64kb).
I have cloned the service onto a Windows Server 2008 (IIS 7.5) as well as the original Server 2003 (IIS 6.0).
Both of these behave the same and return a 'Bad Request" Error.

Fiddler shows:
HTTP/1.1 400 Bad Request
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Mon, 15 Jul 2013 08:54:19 GMT
Content-Length: 0

And IIS error log shows:
Event Name: MODULE_SET_RESPONSE_ERROR_STATUS Warning
Details: ModuleName="ServiceModel", Notification="AUTHENTICATE_REQUEST", HttpStatus="400", HttpReason="Bad Request", HttpSubStatus="0", ErrorCode="The operation completed successfully. (0x0)", ConfigExceptionInfo=""

I can manually create a POST in Fiddler, and it will update a single record OK, provided that the Base64 data is small.
With the exact same POST except a larger encoded image, the Bad Request error occurs.

I have extracted the Base64 encoded image data and been able to decode it to re-produce the large image.
I can also encode any small image and POST it OK, so I am sure that the encoded Base64 image data is valid.

I cannot see any way to workaround this issue, and have spent several days investigating.
Any suggestion you can offer, perhaps a complete 'ServiceModel" section would be helpful?

This is my current full "ServiceModel" section
<system.serviceModel>
    <diagnostics>
        <messageLogging logEntireMessage="true" logMalformedMessages="true" logMessagesAtServiceLevel="true" logMessagesAtTransportLevel="true" maxMessagesToLog="-1"/>
    </diagnostics>
    <bindings>
        <webHttpBinding>
            <binding name="riskbinding" maxReceivedMessageSize="104857600"/>
        </webHttpBinding>
    </bindings>
</system.serviceModel>
The rest of the web.config file is almost the same as the original ListSample config except for the SQL connection.

Thanks for your help so far, I hope that you can offer an insight into this problem.

Regards,
Geoff
Jul 15, 2013 at 11:14 AM
whole web.config:
<?xml version="1.0"?>
<configuration>
  <appSettings />
  <connectionStrings>
    <add name="ConnectionString" connectionString="****" />
  <add name="AMT" connectionString="metadata=*****" providerName="System.Data.EntityClient" /></connectionStrings>
  <system.web>
    <compilation debug="true" targetFramework="4.0"><assemblies><add assembly="System.Data.Entity, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" /></assemblies></compilation>
    <authentication mode="Windows" />
    <customErrors mode="Off" />
    <pages controlRenderingCompatibilityVersion="3.5" clientIDMode="AutoID" />
    <httpRuntime  maxRequestLength="102400" executionTimeout="1000"/>
  </system.web>
  <system.serviceModel>

    <behaviors>
      <serviceBehaviors>
        <behavior name="">
          <serviceMetadata httpGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="false" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <bindings>
    <webHttpBinding>
       <binding name="" maxReceivedMessageSize="104857600"/>
    </webHttpBinding>
    </bindings>
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true" aspNetCompatibilityEnabled="true"/>
  </system.serviceModel>
</configuration>
Jul 15, 2013 at 12:58 PM
Edited Jul 22, 2013 at 1:50 AM
Hi, Selvin.
Thanks for your info. I was receiving the error:
'HTTP/1.1 500 System.ServiceModel.ServiceActivationException'.

Thanks,
Geoff

(Deleted web.config - not valid - service hosting error was due to server missing full configuration for WCF)
Jul 17, 2013 at 12:23 PM
Edited Jul 22, 2013 at 1:51 AM
Hi, Selvin,

Thanks to your guidance, and a lot of work with the WCF functionality, I have achieved synching of large blobs from SQLite to SQL Server.
Woohoo!! :-)

Because I wanted to ensure that it was possible to reliably upload many records with large images, I have extended the sync code to process specific tables one row at a time.
I have also implemented a view functionality, similar to table module, that allows me to add "select a value" starting record to a query result, for use in spinners.

Your sample and help has been very useful to build a working application.
So, in order to contribute back to the community, I am willing to contribute a version of the extension code to your sample project.
Please advise if you are interested in reviewing and perhaps integrating the extension code.

If you would like to communicate outside of the discussion, please advise with a private email and we can continue that way.

I have edited and cleaned my last post, since it was mostly irrelevant to the sync discussion.

Thanks,
Geoff