Recently we were working with Azure Logic Apps to invoke Azure Functions.
By Default, Logic App runs parallel threads and we didn’t explicitly control the concurrency and left the default values.
So Logic App invoked several concurrent threads which in turn invoked several Azure Functions.
The problem was Azure Functions invoked Database Calls which caused Deadlocks. In Ideal world, Database should be able to handle numerous concurrent functions without deadlocks. Our process high percentage of shared data and we wanted to ensure the consistency , so we had Explicit Transactions in our Stored procedure calls. That’s the root cause of the problem and we didn’t want to remove the explicit Transaction.
The solution we implemented to alleviate this problem is to run this process in Sequence instead of parallel threads.
For each loops execute in parallel by default. Customize the degree of parallelism, or set it to 1 to execute in sequence.
Transaction (Process ID 166) was deadlocked on lock
| communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
So we have identified Deadlock happened in the database through our Application Insights. Next logical question is , what caused this deadlock.
These queries identifies the deadlock event time as well as the deadlock event details.
SELECT * FROM sys.event_log WHERE event_type = 'deadlock'; WITH CTE AS ( SELECT CAST(event_data AS XML) AS [target_data_XML] FROM sys.fn_xe_telemetry_blob_target_read_file('dl', null, null, null) ) SELECT target_data_XML.value('(/event/@timestamp)', 'DateTime2') AS Timestamp, target_data_XML.query('/event/data[@name=''xml_report''] /value/deadlock') AS deadlock_xml, target_data_XML.query('/event/data[@name=''database_name''] /value').value('(/value)', 'nvarchar(100)') AS db_name FROM CTE
You can save the Deadlock xml as xdl to view the Deadlock Diagram. This provides all the information we need to identify the root cause of the deadlock and take necessary steps to resolve the issue.
CloudIQ is a leading Cloud Consulting and Solutions firm that helps businesses solve today’s problems and plan the enterprise of tomorrow by integrating intelligent cloud solutions. We help you leverage the technologies that make your people more productive, your infrastructure more intelligent, and your business more profitable.
626 120th Ave NE, B102, Bellevue,
No. 3 & 4, Venkateswara Avenue,Bazaar Main Rd, Ramnagar South, Madipakkam, Chennai – 600091
© 2019 CloudIQ Technologies. All rights reserved.