Showing posts with label Services. Show all posts
Showing posts with label Services. Show all posts

Friday, April 25, 2014

Multi-threading - Dependent tasks in batch using SysOperation framework

For a recent requirement, we had to make a custom batch job that could process thousands of lines coming from retail stores and post sales orders, purchase orders and counting journals for stock adjustments in AX. This batch job was to run on a regular basis daily. And to raise the bar of complexity even further, there was a sequence to the postings. First goes purchase, then sales and finally stock. Plus there were methods at the beginning and at the end for preparing the data and cleaning up activities at the end. So all in all we have a heavy duty batch job doing lots of heavy lifting and at the same time maintaining a sequence. That's when i started looking at multi-threading in AX.

Adding dependent tasks in AX 2009 has been possible using RunBaseBatch framework. Read this post
http://blogs.msdn.com/b/axsupport/archive/2011/04/13/threading-in-dynamics-ax.aspx

But we needed to tweak this approach as we have the SysOperation framework replacing most of the RunBase code going in future. All new batch jobs, ssrs reports rely on SysOperation classes.

I am going to show few of the tweakings we did and how we used them in the batch job.

First of all we needed a way to have a list of tasks that could be executed in parallel and at the same time define a dependent task. We created a custom class that extended SysOperationServiceBase and custom method that took a list and a task(class) as parameter. We called it addRunTimeTaskList().

Notice how we loop through the list of tasks, calling addRunTimeTask() for each and at the end call addDependency(). This is important you can't call addDependency() without calling addRunTimeTask() first. Note addDependency() method makes task _batchTaskAfter a dependent of each batchTask of the list. Enum BatchDependencyStatus::Finished tells that dependent task will only start executing once the parent tasks have finished successfully.

 /// <summary>  
 /// Used to add a list of runtime tasks to the current batch job  
 /// </summary>  
 /// <param name="_batchTasks">  
 /// List of tasks to be added  
 /// </param>  
 /// <param name="_batchTaskAfter">  
 /// task to be run after all other tasks are complete  
 /// </param>  
 /// <remarks>  
 ///  
 /// </remarks>  
 protected void AddRunTimeTaskList(List _batchTasks, Batchable _batchTaskAfter = null)  
 {  
   BatchHeader bh = this.getCurrentBatchHeader();  
   Batch b = this.getCurrentBatchTask();  
   Batchable batchTask;  
   ListEnumerator listEnum;  
   boolean isRunTimeJob = bh.parmRuntimeJob();  
   listEnum = _batchTasks.getEnumerator();  
   listEnum.reset();  
   if( this.isExecutingInBatch() )  
   {  
     if( _batchTaskAfter )  
     {  
       bh.addRuntimeTask(_batchTaskAfter,b.RecId);  
     }  
     /*Now loop through all the tasks and add them*/  
     while(listEnum.moveNext())  
     {  
       batchTask = listEnum.current();  
       //b.RunTimeTask = true;  
       //bh.addTask(batchTask);  
       bh.addRuntimeTask(batchTask,b.RecId);  
       if(_batchTaskAfter)  
       {  
         bh.addDependency(_batchTaskAfter,batchTask,BatchDependencyStatus::Finished);  
       }  
     }  
     try  
     {  
       ttsBegin;  
       //Need to restore this value as there is an issue in this version of AX  
       bh.parmRuntimeJob(isRunTimeJob);  
       bh.save();  
       ttsCommit;  
     }  
     catch( Exception::UpdateConflict )  
     {  
       error("Failed to add child task");  
     }  
   }  
   else  
   {  
     /*Now loop through all the tasks and add them*/  
     while(listEnum.moveNext())  
     {  
       batchTask = listEnum.current();  
       batchTask.run();  
     }  
     /*See if there is a batch to run after*/  
     if(_batchTaskAfter)  
     {  
       _batchTaskAfter.run();  
     }  
   }  
 }  

This is how we call the method addRunTimeTaskList() passing a list and a call to another method as dependent.

 this.AddRunTimeTaskList(batchTasks_Sales, this.batchTaskAfterSales());  

Lets see how we populate batchTasks_Sales. We while through a query each time calling method addUpdateBatch_Sales(), in which we fill a list with instances of SysOperationServiceController class and setting the data contract alongwith.

   while (queryRun.next())  
   {  
     mmsStagingImportTable     = queryRun.get(tableNum(MMSStagingImportTable));  
     if (queryRun.changed(tableNum(MMSStagingImportTable)))  
     {  
       updateBatch.parmDescription(strFmt("@SYS76785", startingPosition, "@MMS2867"));  
       startingPosition++;  
       updateBatch.parmStartDate(processingDate);  
       updateBatch.parmImportId(mmsStagingImportTable.ImportId);  
       this.addUpdateBatch_Sales(updateBatch, classStr(MMSStagingDataSalesCopy), methodStr(MMSStagingDataSalesCopy, copyData));  
     }  
   }  
 public void addUpdateBatch_Sales(MMSStagingDataSalesCopyBatchDC _updateBatch, ClassName _class, MethodName _method)  
 {  
   MMSStagingDataSalesCopyBatchDC dataContract;  
   SysOperationServiceController sosc = new SysOperationServiceController(_class, _method, SysOperationExecutionMode::Synchronous );  
   dataContract = sosc.getDataContractObject();  
   if (dataContract)  
   {  
     dataContract.parmDescription(_updateBatch.parmDescription());  
     dataContract.parmStartDate(_updateBatch.parmStartDate());  
     dataContract.parmImportId(_updateBatch.parmImportId());  
   }  
   batchTasks_Sales.addEnd(sosc);  
 }  
Final piece of the puzzle, dependent method batchTaskAfterSales() which calls the next method.
 private SysOperationServiceController batchTaskAfterSales()  
 {  
   MMSStagingMasterPostingDC     dataContractStock;  
   SysOperationServiceController sosc = new SysOperationServiceController(classStr(MMSStagingDataMasterPosting), methodStr(MMSStagingDataMasterPosting, batchStores), SysOperationExecutionMode::Synchronous);  
   dataContractStock = sosc.getDataContractObject();  
   return sosc;  
 }  
If you run the batch job and have it executing, going to the "View tasks" and selecting dependent task, you could see in the bottom grid the parent tasks.




EDIT: Its important that you use SysOperationExecutionMode::Synchronous in instantiating SysOperationServiceController otherwise your dependent task would run before.

AIF Document Services: Handling header-line relationships and custom operations

Using AIF document services to create data in AX is not a new thing. Its been around for a while now. Document services component of AX is easily the most used aspect when it comes to using AIF services with AX. For most of the basic AX tables, document services are shipped out of the box. For the custom tables that you create, you can easily create document services exposing the tables to the outside world. Create, Read, Update are commonly used service operations. Today i want to discuss how we can use these document services to create some data maintaining the header-line relations of the tables involved. We have two tables, MMSStagingImportTable the parent and MMSStagingStockImport the child and lets assume that we have a query with this relationship and all the Axd* and Ax* objects are already created in AX, port containing the service operations is activated and service reference is added in a .NET console application. For more info refer
http://technet.microsoft.com/en-us/library/aa856656.aspx

Below code is a function loadStock() in the .NET console application that shows how i am creating a header with 2 lines using the create service operation.

     static void loadStock()  
     {  
       // Instantiate an instance of the service client class.  
       MMSIntellipharmServiceClient client = new MMSIntellipharmServiceClient();  
       // Initizlise parameters used for the create method  
       CallContext context = new CallContext() { Company = "mms" };  
       // Create an instance of the document class.  
       AxdMMSIntellipharm stagingForm = new AxdMMSIntellipharm();  
       // Create instances of the entities that are used in the service and set the needed fields on those entities.  
       AxdEntity_MMSStagingImportTable stagingImportTable = new AxdEntity_MMSStagingImportTable();  
       stagingImportTable.ImportId = "IPH_1002";  
       stagingImportTable.ImportDate = new DateTime(2014, 4, 4);  
       stagingImportTable.Datasource = AxdEnum_MMSImportDatasource.IPh;  
       stagingImportTable.Type = AxdEnum_MMSImportType.Stock;  
       AxdEntity_MMSStagingStockImport stagingStockImport = new AxdEntity_MMSStagingStockImport();  
       stagingStockImport.Barcode = "535353";  
       stagingStockImport.Cost = 14;  
       stagingStockImport.IsGST = AxdExtType_MMSImportIsGST.Yes;  
       stagingStockImport.ListPrice = 23;  
       stagingStockImport.SellPrice = 24;  
       stagingStockImport.SOH = 50;  
       stagingStockImport.StockDate = new DateTime(2014, 3, 25);  
       stagingStockImport.StoreId = "1300";  
       stagingStockImport.StoreItemDesc = "desc";  
       stagingStockImport.StoreItemId = "555555";  
       AxdEntity_MMSStagingStockImport stagingStockImport2 = new AxdEntity_MMSStagingStockImport();  
       stagingStockImport2.Barcode = "768756";  
       stagingStockImport2.Cost = 13;  
       stagingStockImport2.IsGST = AxdExtType_MMSImportIsGST.Yes;  
       stagingStockImport2.ListPrice = 27;  
       stagingStockImport2.SellPrice = 26;  
       stagingStockImport2.SOH = 40;  
       stagingStockImport2.StockDate = new DateTime(2014, 3, 24);  
       stagingStockImport2.StoreId = "1301";  
       stagingStockImport2.StoreItemDesc = "desc";  
       stagingStockImport2.StoreItemId = "444444";  
       stagingImportTable.MMSStagingStockImport = new AxdEntity_MMSStagingStockImport[2] { stagingStockImport, stagingStockImport2 };  
       stagingForm.MMSStagingImportTable = new AxdEntity_MMSStagingImportTable[1] { stagingImportTable };  
       // Call the create method on the service passing in the document.  
       EntityKey[] keys = client.create(context, stagingForm);  
       // The create method returns an EntityKey which contains the RecID of the header.  
       EntityKey returnedRecord = (EntityKey)keys.GetValue(0);  
       Console.WriteLine("New stock header created " + returnedRecord.KeyData[0].Value);  
       Console.ReadLine();  
     }  

This function can be called from the main() method like below.

     static void Main(string[] args)  
     {  
       Console.WriteLine("Choose the action you want to perform:");  
       Console.WriteLine("1: Load purchase data");  
       Console.WriteLine("2: Load sales data");  
       Console.WriteLine("3: Load stock data");  
       Console.WriteLine("4: Load queue");  
       Console.WriteLine("5: Load stock data batches");  
       Console.WriteLine("6: Make import Id ready");  
       Console.WriteLine("7: Add ready import Ids to queue");  
       string userChoice = Console.ReadLine();  
       switch (userChoice)  
       {  
         case "1":  
           Program.loadPurchase();  
           break;  
         case "2":  
           Program.loadSales();  
           break;  
         case "3":  
           Program.loadStock();  
           break;  
         case "4":  
           Program.loadQueue();  
           break;  
         case "5":  
           Program.loadStockMultiple();  
           break;  
         case "6":  
           Program.makeReady();  
           break;  
         case "7":  
           Program.addToQueue();  
           break;  
       }  
       Console.WriteLine("Press any key to exit");  
       Console.ReadLine();  
     }  

Next we will enhance the loadStock() method by inserting 2 more lines to the same header. I will be using service operations read and update. I haven't seen any example of this kind yet, which is the primary reason for this blog post. This update is actually not updating anything in our case. It merely reads the returned header from the EntityKey[] and adds two more lines to it. Can you notice the differences between loadStockMultiple() and loadStock()?

     static void loadStockMultiple()  
     {  
       // Instantiate an instance of the service client class.  
       MMSIntellipharmServiceClient client = new MMSIntellipharmServiceClient();  
       // Initizlise parameters used for the create method  
       CallContext context = new CallContext() { Company = "mms" };  
       // Create an instance of the document class.  
       AxdMMSIntellipharm stagingForm = new AxdMMSIntellipharm();  
       // Create instances of the entities that are used in the service and set the needed fields on those entities.  
       AxdEntity_MMSStagingImportTable stagingImportTable = new AxdEntity_MMSStagingImportTable();  
       stagingImportTable.ImportId = "IPH_1005";  
       stagingImportTable.ImportDate = new DateTime(2014, 4, 4);  
       stagingImportTable.Datasource = AxdEnum_MMSImportDatasource.IPh;  
       stagingImportTable.Status = AxdEnum_MMSImportStatus.New;  
       stagingImportTable.Type = AxdEnum_MMSImportType.Stock;  
       AxdEntity_MMSStagingStockImport stagingStockImport = new AxdEntity_MMSStagingStockImport();  
       stagingStockImport.Barcode = "535353";  
       stagingStockImport.Cost = 14;  
       stagingStockImport.IsGST = AxdExtType_MMSImportIsGST.Yes;  
       stagingStockImport.ListPrice = 23;  
       stagingStockImport.SellPrice = 24;  
       stagingStockImport.SOH = 50;  
       stagingStockImport.StockDate = new DateTime(2014, 3, 25);  
       stagingStockImport.StoreId = "1300";  
       stagingStockImport.StoreItemDesc = "desc";  
       stagingStockImport.StoreItemId = "555555";  
       AxdEntity_MMSStagingStockImport stagingStockImport2 = new AxdEntity_MMSStagingStockImport();  
       stagingStockImport2.Barcode = "768756";  
       stagingStockImport2.Cost = 13;  
       stagingStockImport2.IsGST = AxdExtType_MMSImportIsGST.Yes;  
       stagingStockImport2.ListPrice = 27;  
       stagingStockImport2.SellPrice = 26;  
       stagingStockImport2.SOH = 40;  
       stagingStockImport2.StockDate = new DateTime(2014, 3, 24);  
       stagingStockImport2.StoreId = "1301";  
       stagingStockImport2.StoreItemDesc = "desc";  
       stagingStockImport2.StoreItemId = "444444";  
       stagingImportTable.MMSStagingStockImport = new AxdEntity_MMSStagingStockImport[2] { stagingStockImport, stagingStockImport2 };  
       stagingForm.MMSStagingImportTable = new AxdEntity_MMSStagingImportTable[1] { stagingImportTable };  
       // Call the create method on the service passing in the document.  
       EntityKey[] keys = client.create(context, stagingForm);  
       //update >  
       // Use the keys to read   
       AxdMMSIntellipharm stagingFormUpd = client.read(context, keys);  
       // Create instances of the entities that are used in the service and set the needed fields on those entities.  
       AxdEntity_MMSStagingImportTable stagingImportTableUpd = stagingFormUpd.MMSStagingImportTable.First();  
       stagingImportTableUpd.action = AxdEnum_AxdEntityAction.update;  
       stagingImportTableUpd.actionSpecified = true;  
       AxdEntity_MMSStagingStockImport stagingStockImportUpd1 = new AxdEntity_MMSStagingStockImport();  
       stagingStockImportUpd1.action = AxdEnum_AxdEntityAction.create;  
       stagingStockImportUpd1.actionSpecified = true;  
       stagingStockImportUpd1.Barcode = "53535";  
       stagingStockImportUpd1.Cost = 14;  
       stagingStockImportUpd1.IsGST = AxdExtType_MMSImportIsGST.Yes;  
       stagingStockImportUpd1.ListPrice = 23;  
       stagingStockImportUpd1.SellPrice = 24;  
       stagingStockImportUpd1.SOH = 50;  
       stagingStockImportUpd1.StockDate = new DateTime(2014, 3, 25);  
       stagingStockImportUpd1.StoreId = "1300";  
       stagingStockImportUpd1.StoreItemDesc = "desc";  
       stagingStockImportUpd1.StoreItemId = "555555";  
       AxdEntity_MMSStagingStockImport stagingStockImportUpd2 = new AxdEntity_MMSStagingStockImport();  
       stagingStockImportUpd2.action = AxdEnum_AxdEntityAction.create;  
       stagingStockImportUpd2.actionSpecified = true;  
       stagingStockImportUpd2.Barcode = "76875";  
       stagingStockImportUpd2.Cost = 13;  
       stagingStockImportUpd2.IsGST = AxdExtType_MMSImportIsGST.Yes;  
       stagingStockImportUpd2.ListPrice = 27;  
       stagingStockImportUpd2.SellPrice = 26;  
       stagingStockImportUpd2.SOH = 40;  
       stagingStockImportUpd2.StockDate = new DateTime(2014, 3, 24);  
       stagingStockImportUpd2.StoreId = "1301";  
       stagingStockImportUpd2.StoreItemDesc = "desc";  
       stagingStockImportUpd2.StoreItemId = "444444";  
       stagingImportTableUpd.MMSStagingStockImport = new AxdEntity_MMSStagingStockImport[2] { stagingStockImportUpd1, stagingStockImportUpd2 };  
       stagingFormUpd.MMSStagingImportTable = new AxdEntity_MMSStagingImportTable[1] { stagingImportTableUpd };  
       client.update(context, keys, stagingFormUpd);  
       EntityKey returnedRecord = (EntityKey)keys.GetValue(0);  
       Console.WriteLine("New stock header created & updated " + returnedRecord.KeyData[0].Value);  
       Console.ReadLine();  
       //update <  
     }  

The RecVersion value in AX for the header remains 1, which shows that there is no update to the record. Also notice the use of action and actionSpecified with each buffer, in case of header we use AxdEnum_AxdEntityAction.update and in case of lines we use AxdEnum_AxdEntityAction.create.

We can also add custom service operations along with the standard ones as you can see in my main method, addToQueue() and makeReady(). makeReady() expects a parameter.
Both the methods are added to the DocumentService class in AX with attribute [SysEntryPointAttribute(true)].

.NET code:
     static void makeReady()  
     {  
       try  
       {  
         Console.WriteLine("Enter an Import Id to make ready :");  
         string importId = Console.ReadLine();  
         // Instantiate an instance of the service client class.  
         MMSIntellipharmServiceClient client = new MMSIntellipharmServiceClient();  
         // Initizlise parameters used for the create method  
         CallContext context = new CallContext() { Company = "mms" };  
         // Create an instance of the document class.  
         AxdMMSIntellipharm staging = new AxdMMSIntellipharm();  
         client.makeReady(context, importId);  
         Console.WriteLine("Success!");  
       }  
       catch (Exception e)  
       {  
         Console.WriteLine(e.InnerException.Message);  
       }  
       Console.ReadLine();  
     }  

     static void addToQueue()  
     {  
       try  
       {  
         // Instantiate an instance of the service client class.  
         MMSIntellipharmServiceClient client = new MMSIntellipharmServiceClient();  
         // Initizlise parameters used for the create method  
         CallContext context = new CallContext() { Company = "mms" };  
         // Create an instance of the document class.  
         AxdMMSIntellipharm staging = new AxdMMSIntellipharm();  
         client.addToQueue(context);  
         Console.WriteLine("Success!");  
       }  
       catch (Exception e)  
       {  
         Console.WriteLine(e.InnerException.Message);  
       }  
       Console.ReadLine();  
     }  

AX code:
 [SysEntryPointAttribute(true)]  
 public void makeReady(MMSImportID _importId)  
 {  
   MMSStagingImportTable  importTable;  
   ttsbegin;  
   select firstOnly forupdate importTable where importTable.ImportId == _importId;  
   if (importTable)  
   {  
     importTable.Status = MMSImportStatus::Ready;  
     importTable.update();  
   }  
   ttscommit;  
 }  

 [SysEntryPointAttribute(true)]  
 public void addToQueue()  
 {  
   MMSStagingImportTable  importTable;  
   MMSStagingImportQueue  queueTable, queueTableNot;  
   MMSSeqId        seqId;  
   NumberSeq        numberSeq;  
   numberSeq = NumberSeq::newGetNum(MMSStagingImportQueue::numRefSeqId());  
   numberSeq.used();  
   seqId = numberSeq.num();  
   ttsBegin;  
   insert_recordset queueTable(ImportID, SeqID)  
   select ImportID, seqId  
     from importTable  
     where importTable.Status == MMSImportStatus::Ready  
     notexists join queueTableNot  
     where queueTableNot.ImportID == importTable.ImportId;  
   ttsCommit;  
 }  

Thats a lot of code for one post. I hope you learned something new today.

Thursday, May 24, 2012

Services in AX 2012 - working example


In Microsoft Dynamics AX 2012, the Application Integration Framework (AIF) underwent a number of dramatic changes and is now known as Services. There are more number of WCF service hosts and simplified forms and concepts.
There are new terms like Service Groups and Ports (Basic, Enhanced), which we need to understand.
Service group - collection of Services. Why would you collect multiple services under a group? Consider a case of creation of a sales order and customer specified doesn't exist in AX. You will need another service for creation of customer. Both these services can be now clubbed together under one Service group. Also, with a single WSDL, you only need to add one service reference in Microsoft Visual Studio to access all of the objects from the services in the Service group.
Ports - Erstwhile (read AX 2009) local endpoints, channels and endpoints have been consolidated into a port.
Basic port - created automatically as soon as you have deployed your Service group. Also, Basic ports are deployed as WCF services only to the AOS and use the NetTcp adapter.
Enhanced port - created manually. They give you several options around hosting, adapter types, document processing, error handling and security. The choices of adapter types are:
•    File system adapter
•    HTTP- To consume services over the Internet, you must use the HTTP Adapter and host services on IIS. IIS routes all service requests to the AOS. Regardless of the origin of the service request, Internet or Intranet, all the service requests are processed on the AOS.
•    MSMQ - Similar to File adapter. For the URI of the Address and Response addresses browse to the directories that are mail queue directories.
•    NetTcp- Use this adapter when you want to use a service on the Intranet and not on the Internet.

Using the File Adapter
In this example, you will configure an enhanced port to use the file system adapter. Then you will import a sales order using the port.
Overview of the task
1. Create a new enhanced port.
2. Set the Adapter to File system adapter.
3. Set the default file owner to the administrator account.
4. Add the SalesOrder create operation to the list of service operations exposed on this port.
5. Activate the port.
6. Create an xml document that contains a sales order.
7. Run an X++ job that runs the AIF classes and imports the sales order.

Step by Step Instructions
Create and Activate an Enhanced Port
1. In Windows Explorer, create a Services directory and inside of it create an Inbound and an Outbound directory.
2. In Microsoft Dynamics AX under System administration, click Setup, click Services and Application Integration Framework, click Inbound ports and then click New.
3. Enter the name FileSystemAdapterPort for the port.
4. Click Register Adapters to ensure you see all of the adapters available in the environment.
5. Click Register services to ensure you see all of the service operations available in the environment.
6. In the Address section for the Adapter field, select File system adapter.
7. In the Address section for the URI field, click the dropdown and navigate to the Services - Inbound directory.
8. In the Response address section for the Adapter name, select File system adapter.
9. In the Response address section for the URI field, click the dropdown and navigate to the Services - Outbound directory.

10. In the Address section, click Configure.
11. Mark the Enable default file owner and select the Admin user. Admin will be the default owner of the xml files submitted to the Inbound directory.
12. Click Services operations.
13. From the Remaining service operations list, select SalesSalesOrderService.create and click the < arrow.
14. Click Close.
15. Click Activate.
16. Close the ports form. 
 
Create a XML File

1. In Windows, open Notepad.
2. Copy the following to a new Notepad document: (Note: Not able to display the below XML code in the blog, somehow displays junk characters, workaround i found is to use # in front of each row. Take care to remove the # in your actual code.)
# 
# 
#
#northamerica\dyndust #CEU #http://schemas.microsoft.com/dynamics/2008/01/services/SalesOrderService/create #
#Body # # # #DAT-000001 #2011-01-16 #en-us #PO123 # #1001 #88.00 #ea # #HD #01 #2 #42 # # # # # #/Body #
 
3. Save the file in the Services - Inbound directory. Name the file SalesOrder.xml.

Create a Job

1. In the AOT, create a new job with the following code in it:

AifGatewayReceiveService agrs = new AifGatewayReceiveService(); 
AifInboundProcessingService aip = new AifInboundProcessingService(); 
AifOutboundProcessingService aop = new AifOutboundProcessingService(); 
AifGatewaySendService agss = new AifGatewaySendService(); 
; 
agrs.run();
aip.run(); 
aop.run(); 
agss.run();


2. Save the job.
3. Execute the job.

Verify the Results
 In AX, navigate to Sales and marketing, click Common, click Sales orders and then click All sales orders. You should have a new order created for the customer account 1101. If you do not have a new sales order in Microsoft Dynamics AX, navigate to System administration, click Services and Application Integration Framework and then click Exceptions to see the exceptions logged.