The DR650GW-2CH like the DR600GW and the Blackvue DR750LW-2CH cannot recognise exFAT format SDXC cards, in order to get the Blackvue DR650GW-2CH to boot up successfully with a 64GB Micro SDXC card inserted you need to format card to FAT32.
To fix this problem, use the BlackVue viewer app and the format function within it.
Sunday, 20 March 2016
Thursday, 4 February 2016
At Least Once Messaging, Atomic operations and SQL Server
In a NServiceBus system with at-least-once-messaging, there is always the chance that two handlers are executed with identical copies of a message. Handlers should be idempotent and protect against the same message producing different outcomes.
When we move deeper down to the repository layer, it raises interesting implications. If we have an Upsert style Stored Procedure (Update/Insert), then should this protect against high concurrency situations?
Take the excerpt below from a financial system:
The Select then Insert/Update semantic above is not protected against concurrent access and this has a [low] potential of happening with our messaging infrastructure.
If two handlers process a copy of the same message we have the potential to do two inserts into the PaymentLedger table.
This article discusses the problem and suggests a fix.
Even the merge command is not immune from this problem. It also demonstrates a good method for replicating the problem by scheduling a SQL command.
We can rely upon the caller above us to protect us from this scenario. However if we want the Stored Procedure to be explicitly safe against high-volume concurrency then the transaction really should be applied at the SQL level and not reply upon the client applying it.
We may not trust our client because
An example fix is to modify the SQL as follows:
This change will affect performance very slightly by reducing the throughput of these inserts/updates.
When we move deeper down to the repository layer, it raises interesting implications. If we have an Upsert style Stored Procedure (Update/Insert), then should this protect against high concurrency situations?
Take the excerpt below from a financial system:
IF NOT EXISTS (SELECT ReceiptId FROM [dbo].[PaymentLedger] WHERE ReceiptId=@ReceiptId) BEGIN INSERT INTO [dbo].[PaymentLedger]
The Select then Insert/Update semantic above is not protected against concurrent access and this has a [low] potential of happening with our messaging infrastructure.
If two handlers process a copy of the same message we have the potential to do two inserts into the PaymentLedger table.
This article discusses the problem and suggests a fix.
Even the merge command is not immune from this problem. It also demonstrates a good method for replicating the problem by scheduling a SQL command.
We can rely upon the caller above us to protect us from this scenario. However if we want the Stored Procedure to be explicitly safe against high-volume concurrency then the transaction really should be applied at the SQL level and not reply upon the client applying it.
We may not trust our client because
- may have not applied an idempotency check correctly
- our endpoints often don’t wrap the SQL in a transaction
- or have disabled distributed transactions.
An example fix is to modify the SQL as follows:
BEGIN TRAN
IF NOT EXISTS (SELECT ReceiptId FROM [dbo].[PaymentLedger] WITH (UPDLOCK, SERIALIZABLE) WHERE ReceiptId=@ReceiptId)
BEGIN
INSERT INTO [dbo].[PaymentLedger]
....
END
COMMIT TRAN
This change will affect performance very slightly by reducing the throughput of these inserts/updates.
Friday, 29 January 2016
ASP.NET MVC WebApi returned a 404 on a Windows 2003 R2 32-bit server
We promoted an ASP.NET MVC WebAPI project from Dev to Production.
It was working fine in dev and we XCOPYd the files from Dev to Production.
All of the IIS settings were comparable - or so we thought - so what can go wrong?
Well, the famous 404 error can. We were getting a 404 on all requests to the Web API.
We tried various things, like setting the
<modules runAllManagedModulesForAllRequests="true" />
element.
We did an aspnet_regiis.exe -i.
I checked we didn't have to allow 32-bit applications on a 64-bit machine.
We tried setting up a wildcard script map.
None of these worked. Nothing was showing in Event Viewer.
In the end we went to the IIS logs. We noticed the 404 code, but also a subcode 2 and sc-substatus of 1260. This article showed that it was a lockdown policy that prevented the request and the 1260 code indicated ERROR_ACCESS_DISABLED_BY_POLICY.
This was because under the Web Service Extensions branch in IIS Manager, the ASP.NET v4.0.30319 was set to prohibited.
Lesson learned - check the IIS log and look at the detailed error codes. The 404 has a subcode and the statuscode yields even more information.
It was working fine in dev and we XCOPYd the files from Dev to Production.
All of the IIS settings were comparable - or so we thought - so what can go wrong?
Well, the famous 404 error can. We were getting a 404 on all requests to the Web API.
We tried various things, like setting the
<modules runAllManagedModulesForAllRequests="true" />
element.
We did an aspnet_regiis.exe -i.
I checked we didn't have to allow 32-bit applications on a 64-bit machine.
We tried setting up a wildcard script map.
None of these worked. Nothing was showing in Event Viewer.
In the end we went to the IIS logs. We noticed the 404 code, but also a subcode 2 and sc-substatus of 1260. This article showed that it was a lockdown policy that prevented the request and the 1260 code indicated ERROR_ACCESS_DISABLED_BY_POLICY.
This was because under the Web Service Extensions branch in IIS Manager, the ASP.NET v4.0.30319 was set to prohibited.
Lesson learned - check the IIS log and look at the detailed error codes. The 404 has a subcode and the statuscode yields even more information.
Sunday, 13 December 2015
Electronic Arts have the worst support portal EVER!
I'm incredulous at EA's very bad support process.
I had purchased some in-app content for my son's tablet, and after an app reinstall it would not resume the content.
So I tried, repeat, tried to contact EA.
Their Contact Us portal is horrific. It is a "fake" contact us, wherein it pretends to offer you an ability to contact but in reality forces you down the check forums and raise a ticket via the portal.
But the portal was horrific.
Trying to raise a new ticket on this URL didn't work.
1. Go to https://help.ea.com/en/fifa/ea-sports-fifa/
2. Scroll down and select "Contact Us"
3. Click on Android Tablet and select Next
4. Select "Missing Content" and "Had content and lost it"
5. In the "Please tell us a little more about your issue" box type "In-App Purchases"
6. Press Next
7. You get dumped on Step 3 above. You cannot complete the support request.
Eventually I figured I had to raise a ticket under the wrong option and submit it. And then attach the JPG after.
I had purchased some in-app content for my son's tablet, and after an app reinstall it would not resume the content.
So I tried, repeat, tried to contact EA.
Their Contact Us portal is horrific. It is a "fake" contact us, wherein it pretends to offer you an ability to contact but in reality forces you down the check forums and raise a ticket via the portal.
But the portal was horrific.
Trying to raise a new ticket on this URL didn't work.
1. Go to https://help.ea.com/en/fifa/ea-sports-fifa/
2. Scroll down and select "Contact Us"
3. Click on Android Tablet and select Next
4. Select "Missing Content" and "Had content and lost it"
5. In the "Please tell us a little more about your issue" box type "In-App Purchases"
6. Press Next
7. You get dumped on Step 3 above. You cannot complete the support request.
Eventually I chose a different selection option and got to the end screen. But if I attached a PNG to the ticket whilst raising it - it dumped you back at the beginning.
Eventually I figured I had to raise a ticket under the wrong option and submit it. And then attach the JPG after.
During this process I was in an email conversation with a support guy who was adamant I should attach the PNG to a support ticket that he had raised of which I had no access to.
In all, this whole debacle cost me 1.5 hours of my life, to fix a problem with my son's in app purchase. It is enough to put me off in-app purchases for life!
NB: I contacted Google and they refunded it within 5 minutes. They've got the right customer experience.
Monday, 7 December 2015
Port exhaustion on Windows server
We had a set of servers that started showing what seemed to be random connection errors. They ran across services and across servers.
Our first thought was it was a hardware or centralised switch problem, but the network guys assured us the switch had no problem.
Then we started leaning towards configuration problems on the boxes. These servers ran multiple services, all heavy socket consumers.
Whenever a socket connection is made to a known port, Windows also uses a temporary socket for the return communications. These temporary sockets, known as ephemeral or short-lived sockets, are assigned from a pool which can be configured.
netsh int ipv4 show dynamicportrange tcp
Protocol tcp Dynamic Port Range
---------------------------------
Start Port : 50001
Number of Ports : 255
This is an incredibly low number for a server high in comms. It shows that when a socket is created, only 255 are available. Bearing in mind when the communication has completed the socket remains open for a short while (known as the TcpTimedWaitDelay) for efficiency in reuse. This defaults to 30 seconds.
Also the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters showed a MaxUserPort of 1279, which conflicted with the StartPort + Number of Ports range.
In a state of port exhaustion MS documentation says that you can get non-specific errors that just look like general connection errors.
e.g.
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send.
References
https://msdn.microsoft.com/en-us/library/aa560610(v=bts.20).aspx
http://www.outsystems.com/forums/discussion/6956/how-to-tune-the-tcp-ip-stack-for-high-volume-of-web-requests/
https://technet.microsoft.com/en-us/library/cc938219.aspx
Our first thought was it was a hardware or centralised switch problem, but the network guys assured us the switch had no problem.
Then we started leaning towards configuration problems on the boxes. These servers ran multiple services, all heavy socket consumers.
Whenever a socket connection is made to a known port, Windows also uses a temporary socket for the return communications. These temporary sockets, known as ephemeral or short-lived sockets, are assigned from a pool which can be configured.
netsh int ipv4 show dynamicportrange tcp
Protocol tcp Dynamic Port Range
---------------------------------
Start Port : 50001
Number of Ports : 255
This is an incredibly low number for a server high in comms. It shows that when a socket is created, only 255 are available. Bearing in mind when the communication has completed the socket remains open for a short while (known as the TcpTimedWaitDelay) for efficiency in reuse. This defaults to 30 seconds.
Also the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters showed a MaxUserPort of 1279, which conflicted with the StartPort + Number of Ports range.
In a state of port exhaustion MS documentation says that you can get non-specific errors that just look like general connection errors.
e.g.
System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send.
References
https://msdn.microsoft.com/en-us/library/aa560610(v=bts.20).aspx
http://www.outsystems.com/forums/discussion/6956/how-to-tune-the-tcp-ip-stack-for-high-volume-of-web-requests/
https://technet.microsoft.com/en-us/library/cc938219.aspx
Sunday, 6 December 2015
Installing gpiozero on the Raspberry Pi (Wheezy)
I found it troublesome to install the gpiozero package in order to interface to the Rasbperry PI IO.
In the end the following commands worked:
sudo apt-get update
sudo apt-get install python3-pip
sudo pip-3.2 install gpiozero
Test it with
sudo python3 -c "import gpiozero"
https://docs.google.com/document/d/1EbbVjdgXbKVPFlgH_pEEtPZ0zOZVSPHT4sQNW88Am7w/edit?pli=1#
Running the program within IDLE raised the exception about not able to access /dev/mem. To fix this I added the IDLE shortcut to the desktop, ran the nano editor under sudo to edit the shortcut and ran IDLE with the sudo prefix.
In the end the following commands worked:
sudo apt-get update
sudo apt-get install python3-pip
sudo pip-3.2 install gpiozero
Test it with
sudo python3 -c "import gpiozero"
https://docs.google.com/document/d/1EbbVjdgXbKVPFlgH_pEEtPZ0zOZVSPHT4sQNW88Am7w/edit?pli=1#
Running the program within IDLE raised the exception about not able to access /dev/mem. To fix this I added the IDLE shortcut to the desktop, ran the nano editor under sudo to edit the shortcut and ran IDLE with the sudo prefix.
Tuesday, 24 November 2015
Fakes Mocks and Stubs
Fakes, Mocks and Stubs offer varying depth of business logic.
A Fake is a simple object that contains no business logic. It simply contains specific values (such as good or bad) that can be injected into a test to determine the result.
A Mock allows the test to replace a method call or property accessors to return a particular value. For example the test can mock a repository class to return pre-canned data when the Read() method is called. Mocks are also useful in unit tests as they can tested that particular methods were called and how many times they were called. You can assert against a mock.
A Stub is more comprehensive and at the coding level can replace an entire class. For the repository example the repository could be replaced with a StubRepository and this either returns a response based upon input data or based upon some preconfigured data.
Beyond the coding level a Stub can also be used to replace a subsystem. If a payments system is calling a third party interface, it may be necessary to stub out the 3rd party. This benefits if calling the third party costs money, if performance tests are required, if you want to avoid overloading the third party, or if the third party is unable to provide the failure scenarios when you need them.
For a stubbed interface it is often useful to return output data based upon input data. For a Fraud-checking stub, the response may vary based upon the applicant's last name, e.g. fail fraud checks if the last name is "MyLastName_Fail". You want to avoid statefulness in the stub or vary the response from the stub based upon configuration, which would require a system restart.
For performance testing it may also be useful to simulate latency based upon input data. In our Fraud Check example the response may delayed by 5 seconds if the applicant's first name was "MyFirstName_5s". This delay may help tease out scalability problems or race conditions in neighbouring parts of the system. The test driver can then issue a variety of load conditions simply by supplying varying usernames as request parameters.
A Fake is a simple object that contains no business logic. It simply contains specific values (such as good or bad) that can be injected into a test to determine the result.
A Mock allows the test to replace a method call or property accessors to return a particular value. For example the test can mock a repository class to return pre-canned data when the Read() method is called. Mocks are also useful in unit tests as they can tested that particular methods were called and how many times they were called. You can assert against a mock.
A Stub is more comprehensive and at the coding level can replace an entire class. For the repository example the repository could be replaced with a StubRepository and this either returns a response based upon input data or based upon some preconfigured data.
Beyond the coding level a Stub can also be used to replace a subsystem. If a payments system is calling a third party interface, it may be necessary to stub out the 3rd party. This benefits if calling the third party costs money, if performance tests are required, if you want to avoid overloading the third party, or if the third party is unable to provide the failure scenarios when you need them.
For a stubbed interface it is often useful to return output data based upon input data. For a Fraud-checking stub, the response may vary based upon the applicant's last name, e.g. fail fraud checks if the last name is "MyLastName_Fail". You want to avoid statefulness in the stub or vary the response from the stub based upon configuration, which would require a system restart.
For performance testing it may also be useful to simulate latency based upon input data. In our Fraud Check example the response may delayed by 5 seconds if the applicant's first name was "MyFirstName_5s". This delay may help tease out scalability problems or race conditions in neighbouring parts of the system. The test driver can then issue a variety of load conditions simply by supplying varying usernames as request parameters.
Subscribe to:
Posts (Atom)