Deploying Guest Executables to Azure Service Fabric
One of my most popular blog posts on this site is Hands-on with Azure Service Fabric Reliable Services. Referring to the blog post, many readers asked me about guidance to deploy various types of applications as Guest Executables on Service Fabric. In this post, I will try to answer most of those questions by demonstrating the process in a simple manner.
We previously built a simple node.js application that uses Azure SQL Database as data store. I will demonstrate how we can deploy that application to Service Fabric as a Guest Executable. However, none of the steps that I am going to follow are specific to node.js. You can select an application built using any other programming language e.g. Java and follow the same steps to deploy your application (with some changes depending on your application) as a Guest Executable on Service Fabric.
One thing to be cognizant of while selecting an application for the purpose is that your application should be stateless in nature because Guest Executables are deployed as stateless services, i.e. an instance of your application will run on every node in the Service Fabric cluster.
I understand that you need sample code to refer to (a.k.a copy & paste). Here is the sample application that you can clone or download.
I recommend using Visual Studio to package your application for the ease that it offers. However, you can manually package your application as well. Either of the approaches will create a folder structure that is similar to the following:
|-- ApplicationPackageRoot |-- GuestServicePkg |-- Code |-- Application Files |-- Config |-- Settings.xml |-- Data |-- ServiceManifest.xml |-- ApplicationManifest.xml
To create a Guest Executable package using Visual Studio, create a new Service Fabric project and select Guest Executable from the dialog that follows:
Let’s fill the required values in this dialog.
- Code Package Folder: You need to supply the path to the folder where your application binaries live. None of the files outside this folder should contain your application binaries as they will not be deployed to Service Fabric.
- Code Package Behavior: For most of the cases, you should set this option to Add link to external folder. That way only a link to the folder that you previously specified would exist and files won’t be copied to the project that you are currently creating. This helps when you need to redeploy the application at a later point of time with updated code.
- Program: You can specify the name of the program (an exe, bat, ps1 etc.), present inside the folder that you previously specified, that should be run to launch your application. You may leave this field empty for now.
- Arguments: The arguments that should be passed to the program above. It can be a list of parameters with arguments. You may leave this field empty for now.
WorkingFolder: The folder that you specify here becomes the working directory for the Guest Executable process that is going to be started. You can specify one of three values here:
CodeBase: Sets the working directory to the Code folder (or whatever is the name of the code folder). I have selected this option for the application that I am going to deploy.
CodePackage: Sets the working directory to GuestServicePkg (or whatever is the name of the package folder).
Work: Sets the working directory to a subdirectory called work. If you choose this option, you will need to create this folder and place all your binaries inside it.
You might need to install certain dependencies on the node to make your application work. Such dependencies may include installing servers or other software that your application uses. If this is the case, create a script (ps1, cmd, bat etc.) that downloads and installs these dependencies on a VM. You should test the script on a VM several times to check whether the script executes successfully and requires no manual input.
I generally use a package manager to carry out dependency installations. Chocolatey is one of the popular package managers that can take care of installing dependencies on a Windows VM for you. You can install Chocolatey on your VM by invoking this script and later install components on VM using it.
If you can’t find the necessary packages on your favorite package manager, you may also include the components in the code folder and refer the components from your installer script, as the resources will be available in the deployed package.
Launching The application
After installing the dependencies, you would want your application to be launched. Prepare another script that Service Fabric can use to launch your application. This script would be executed after the script that was used to install dependencies. Again, test this script multiple times on a VM to ensure that by running the script, your application does launch successfully.
Installing the dependencies or launching the application may require elevated privileges. I feel it is better to keep this option turned on rather than face incomprehensible errors at startup. To set this up, you need to add a user to the Administrators system group and set a policy to run the scripts using the created user’s principal. Navigate to ApplicationManifest.xml and add the following section to the file to add a user named SetupAdminUser to the Administrators group.
<Principals> <Users> <User Name="SetupAdminUser"> <MemberOf> <SystemGroup Name="Administrators" /> </MemberOf> </User> </Users> </Principals>
Now add a
RunAs policy to the manifest that ensures that all scripts execute under the SetupAdminUser’s principal. You can change the
Main, if you don’t want to execute all scripts with elevated privileges.
<Policies> <RunAsPolicy CodePackageRef="Code" UserRef="SetupAdminUser" EntryPointType="All" /> </Policies>
In this file, you prepare the environment for your application. Let’s configure the elements in this file.
- The UseImplicitHost Setting: This setting tells Service Fabric that the service is based on a self-contained app, so all Service Fabric needs to do is to launch it as a process and monitor its health.
- SetupEntryPoint: If you have any installations that you need to carry out (see above), you need to write the name of the script and its arguments here. Note that this script would be invoked every time the node restarts, therefore this script needs to be idempotent.
<SetupEntryPoint> <ExeHost> <Program>YOUR SCRIPT NAME</Program> <Arguments>ARGUMENTS</Arguments> <WorkingFolder>CodePackage</WorkingFolder> </ExeHost> </SetupEntryPoint>
<EntryPoint> <ExeHost> <Program>node.exe</Program> <Arguments>server.js</Arguments> <WorkingFolder>CodePackage</WorkingFolder> </ExeHost> </EntryPoint>
- Endpoints: If your application requires any endpoints, you can specify them here. My application responds to requests on port 3000. So, I will specify the port by adding an endpoint in the
<Endpoints> <Endpoint Name="External" Port="3000" Protocol="http" Type="Input" UriScheme="http" /> </Endpoints>
Deploy the application on the emulator to validate the settings. If you messed up something somewhere, you may get to see a horrible message in SFX (Service Fabric Explorer) without context.
Error event: SourceId=‘System.Hosting’, Property=‘CodePackageActivation:Code:EntryPoint’. There was an error during CodePackage activation.The service host terminated with exit code:1
If you get stuck with this error, there are a couple of things you can do.
- Logging: Add
ConsoleRedirectionin ServiceManifest.xml to log console output (both stdout and stderr) to a working directory. After you have done so, in SFX you can navigate to Nodes > _Node_N > fabric:/GuestApplication and find Disk Location on the right-hand side. In the folder, you will find a logs folder that contains the logs generated by your application.
- Event Log: Service Fabric logs events in the event log which you can filter and read.
- PerfView: You can collect ETW logs using PerfView if you are using .net. Run it, go to “Collect -> Collect”. De-Select “Merge”. Click “Start Collection”. Now kill your service fabric service in task explorer. Moments later, Service Fabric will start it again. After it failed, “Stop collection” in PerfView. Now double-click on “Events” in the left tree - this will open all recorded ETW events. Search for “Microsoft-Windows-DotNETRuntime/Exception/Start” and double click on it. You should see all .NET exceptions that occurred, ordered by time.
on My System on Cloud
I deployed my application to my Service Fabric cluster on Azure and here is the landing page.