Azure Webjobs Part 2 – building an Azure Webjob

microsoft azure logo

As I mentioned in part one of this 2-part “straight to TV” drama, Azure Webjobs are pretty awesome! They’re Scheduled Tasks without the secrecy. They’re Azure Worker Processes without the cost! You can host them on Azure for free as a Web App (recently renamed from “Websites”) and straight out of the box you get an awesome Dashboard that allows you to check the status of your Webjob and replay any single invocation at a single click.

WebJobs are amazing for doing processes that take a decent chunk of time and processing power, so you don’t hold up the your website’s thread and slow down any responses. For the WebJob that I recently wrote, I dump the url and database Id of a pdf file into the Message Queue, and the WebJob grabs the PDF, creates a thumbnail of the first page, saves it back to my Azure Blob Storage container and adds the thumbnail url back into the database – all for free and without freezing up the UI of my CMS.

In part one I showed you how to add messages to an Azure Message Queue. In this post I will show you how you can build an Azure Webjob to process and do something with those messages.

You will need to have the latest Azure SDK installed, which you should have done already if you followed part one. Oh you didn’t read part one yet? Shame on you…

Set up a WebJob project

First we need to create a new project – in Visual Studio, choose ‘Create New Project’ and choose ‘Azure WebJob’ from the options:

create-new-webjob

You can see this project looks pretty much like a console app – there’s a Program.cs file and an App.config file just like you would expect in a Console Application. The slight difference here is that there is a Functions.cs file. It’s this Functions.cs file that contains the method that will be triggered when a message is added to your message queue (the auto-generated comments on the method tell you pretty much the same thing):

functions-cs

ProcessMessageQueue parameters

You can see there are two parameters to this ProcessMessageQueue method. The first parameter is the actual message that you will be expecting from the Message Queue. Out of the box this is just a simple string, but as I demonstrated in part one it can be any object type you want. You just need to ensure that whatever object type you add to the message queue matches the object type you specify as this ProcessMessageQueue parameter. You will also notice that the message parameter is decorated by a [QueueTrigger(“queue”)] attribute. This is the bit that triggers this function when data is added to the Message Queue. You will need to swap the “queue” text in there to match the Message Queue name you already specified (remember that this needs to be all lower case in order for this to work…

The second parameter TextWriter allows you to write to the WebJob logfile, the contents of which you can view through the Dashboard (more to come on that later) in the same way you would use console.log() in a Console Application.

Add your Storage Account Connection Strings

The only other thing you need to do in order to get this basic WebJob example running is to tell it how to connect to your Message Queue. In order to do this, you need to add your Azure Storage Connection String (which I told you how to find in part one) into the App.config file. Out of the box there is already a placeholder Connection String in the App.config, so all you need to do is add your own data in there:

connection-strings

You will notice there are two Connection String placeholders in the App.config file. The AzureWebJobsStorage Connection String is the one you want to configure at this point. We’ll get on to the AzureWebJobsDashboard Connection String a bit later.

It’s that simple!

Yes it really is that simple to create a WebJob – now you just have to stick whatever code you want inside your ProcessMessageQueue method, so you can process the data that you stuck in the Message Queue in the first place. You can make that bit as simple or as complex as you like!

Publishing your WebJob

Publishing a WebJob to an Azure Web App is pretty simple too – right click on your WebJob project and choose ‘Publish as Azure Webjob…’:

publish-as-webjob

On the next screen that pops up you will see a whole load of options for when and how your WebJob should run (I have chosen ‘Run Continuously’ for this demo) and click OK.

Next you will be able to choose your publish target – select ‘Microsoft Azure Websites’:

publish-target

Next you will be asked to Sign In with your Azure account, and then you get to choose either an existing website or create a new one:

select-website

Choosing ‘New’ will take you to a screen where you can choose a name for your website, select which region you want to host it in and create a new database if you want to:

create-a-site

Now click ‘Create’ and your website will be created for you, and the publishing settings for the site will automatically be downloaded into your publishing profile:

publishing-profile

Finally, click ‘Publish’ to publish your WebJob. The first time you publish a WebJob it takes a while, as it’s uploading all the DLLs etc that you’ve created with your code. Any subsequent publishes all seem to be pretty quick so I assume it’s just pushing up any files that have changed since the last publish.

Debugging your WebJob

So now we have published the WebJob it’s time to discover a pretty awesome feature – you can attach your local Visual Studio to your remote WebJob and debug it in real time! In Visual Studio view the Server Explorer (View > Server Explorer) and you will see ‘Azure’ showing at the top. Expand that and you will be able to see your Storage accounts and your Websites, along with all the other areas of Azure.

Expand the Websites node and you will see your WebJob appear there as a website. Drill down into this website (WebJobs > Continuous) and you will see your WebJob. Right click on your WebJob and select ‘Attach Debugger’:

attach-debugger

Stick a breakpoint in your ProcessQueueMessage method and we will now trigger the WebJob by adding data manually to the Message Queue. Again in Server Explorer, navigate to your Message Queue (if it doesn’t yet exist, you can just right click on the Queues node and create it):

storage-queue

Opening up the Message Queue shows an empty table of messages. There is a set of icons on the top right of this table – click the ‘Add Message’ icon:

add-message

You will see a dialog box pop up – add some text in here (if your WebJob is expecting a more complex object, stick some Json in here to represent that object):

add-message-text

You will see the message added to the Message Queue table:

message-queue

Now just sit back and relax for a few moments, and wait for your WebJob to be triggered. Cross your fingers and toes and shortly you should hit the breakpoint you added to your ProcessMessageQueue method (you did add a breakpoint didn’t you?):

breakpoint

Yes it really is that simple to set up a WebJob!

The Dashboard

There’s only one thing left to show you before I go to bed, and that’s how to get the Dashboard up and running.

In Server Explorer if you right click on your WebJob and click ‘Dashboard’ you will probably see something like this:

dashboard-doesnt-work

If you can’t be bothered to read all that text I’ll save you the task. Basically you haven’t yet set up the Connection String for your Dashboard. You need to set this in 2 places – in your App.config file and also in your WebJob ‘Configure’ section in the Azure Management Portal.

To set the Connection String in your App.config, simply add the Storage Account info in the AzureWebJobsDashboard Connection String (you can use the same Storage Account that you used for the Message Queue, or set up a new one if you wish).

In the Azure Management Portal, choose your WebJob and click on ‘Configure’ from the top menu. Scroll towards the bottom of the page and you will see the Connection Strings section. Add your Connection String data in there. You will need to post that entire long string into the ‘Values’ textbox – DefaultEndpointsProtocol=https;AccountName=…;AccountKey=…) and choose ‘Custom’ from the dropdown menu on the right:

azure-portal-connection-string

Now re-publish your WebJob with that extra config setting, and refresh the Dashboard page again. It should look a bit more like this:

dashboard-works

This Dashboard is pretty cool – it tells you the status of the WebJob, it shows you the logfile output of the WebJob (click the ‘Toggle Output’ button to see the log messages). It also allows you to click on the function that you just invoked (the line at the bottom of this image) and view the details of that invocation (click the ‘Toggle Output’ button again and you will see the line of text we wrote to the logfile):

invocation-details

Finally you can also replay this invocation by hitting the ‘Replay Function’ button at the top of the screen which re-submits that message back to the Message Queue for you. This is really useful if the function fails, and you want to try it again to either debug it or run it once you have fixed a bug.

So that’s a really quick introduction to creating WebJobs – I hope I’ve covered all the basics and you can get up and running with this awesome Azure feature! Let me know in the comments if you’ve run into any issues setting this up, and I will try to help out if I can!

Bedtime – good night!

Posted in Code, Code samples, Web development, Windows Azure Tagged with: , , , ,

8 Trello Shortcuts that will improve productivity!

trello-icon

I use Trello for everything! From managing collaborative software projects to keeping personal To-Do lists, to tracking my weight loss! It’s an awesome tool and I love the way it visually helps me to plan and prioritise my life! Furthermore I’m a very impatient person with not enough time to do all the things I want! So I’m constantly trying to find little shortcuts here and there that will shave off a few seconds or minutes from the task I’m doing, so I can spend that time doing more stuff!

I was chatting to another developer the other day (we do talk to each other occasionally you know…) and I mentioned a couple of Trello shortcuts that save me those few precious seconds. He didn’t know they existed and he was pretty excited about – he assured me he was going to try them out as soon as he got home!

Maybe he was telling the truth, or maybe he was just humouring me and hoping I would just stop talking to him… either way I thought I would share my favourite Trello shortcuts with you, and you can see for yourself how much they improve your Trello productivity!

Shortcut 1 – Navigate to board

change trello board

I currently have 14 boards in my Trello and I will switch backwards and forwards between at least 5 of them in any given day. This was the first shortcut I found and I love it! Press B and then start typing the name of your board. Watch with awe as your list of boards appears and is filtered down to the exact board you are looking for. Then hit Enter, lean back and enjoy those extra few seconds you have saved by not having to click on the Boards list, scroll down to find your board and click on it!

Shortcut 2 – Filter your lists

filter your trello list

If you have long lists and you’re having trouble finding the card you want, press F and start to type. Magically see every list filtered so you can find your card without having to scroll…

Free bonus tip: Press F and then * and you will see the number of cards displayed at the top of each list!

Shortcut 3 – Add a new list

add a new Trello list

I found this one totally by accident one day – if you’re in a board and you double click anywhere on the background, the ‘Add List’ dialogue appears and guess what – you can ADD A NEW LIST!!!

Shortcut 4 – send your board an Email!

email your Trello board

I’m constantly sending myself email reminders to do things or links to articles I want to read but maybe later. So imagine my glee when I discovered that you can send an email to your Trello board and it will turn your email into a card for you!

Expand the menu bar on the right hand side of the screen, and then click on the “Menu” dropdown at the top of that bar. You will see a link that says “Email-to-board Settings” which contains a randomly-generated email address that you can use to send an email to your board. You can choose which list to create the card in, and where you want it to go in that list.

Each board has it’s own email address for each member that has access to it, and you can regenerate your email address (like I did after taking that screenshot just in case you were wondering…) to ensure that only you can email your specific board.

The subject of the email will become the title of the card, and the body of the email will be the description. There are more formatting tricks skillz you can learn here if you want to become an ’email-to-board’ Ninja!

Shortcut 5 – Toggle side menu

toggle your menu

In one of my Trello boards I have 11 lists! Even on my HD laptop screen means I have to scroll my board to see all my lists, and the right-hand side menu bar just gets in the way! But that’s a simple problem to solve – just press W and the side menu will appear or disappear! Nice.

Shortcut 6 – Select a label

select a label

Labels are a great way of categorising cards and giving a visual cue as to which cards belong to what category. When you’re on a card, press L and the Lists menu will magically appear. Each label has a number in brackets next to it. Simply press the number for the label you want, hit Enter and your card will be categorised!

Shortcut 7 – Colour-code your boards

colour-code your Trello boards

If you have a lot of boards and they’re all the standard “Trello Blue” it can be difficult to find the one you want quickly (unless you’re using shortcut 1 that is!) From the right-side menu choose ‘Change Background’ and you will see a small number of colours to choose from (you can get more colours/backgrounds if you’re a paying customer, so if you’re aesthetically inclined, you might want to part with your cash!)

Shortcut 8 – Star your boards

star your boards

Finally if you have a lot of boards in your Trello and you’re using it through an app, starring your favourite boards moves them all to the top of the Boards list/page and makes it much quicker for you to find the board you want!

Bonus Shortcut – set Trello as your browser homepage!

If you’ve got this far you deserve a reward, so here’s a little bonus shortcut! I set my browser homepage to be my ‘Backlog’ Trello board – this is my prioritised backlog where I bring cards in from all my other boards and stack-prioritise them! So each morning when I open my laptop I can see exactly what my priorities are for that day!

So there you go – if you’re using Trello a lot and you use all these shortcuts I can guarantee that you will save at least 2 extra minutes per day! Just think how many tweets you could send with all that saved time!

If you aren’t using Trello yet, use this link to sign up (it’s free!)

Do you have any other Trello shortcuts that you find invaluable? If so, share them in the comments below!

Posted in Productivity, Software, Tips Tagged with: , , ,

Azure Webjobs part 1 – using Azure Storage Message Queues

microsoft azure logo

Azure WebJobs are pretty amazing! They’re similar to Azure Worker Processes but with a couple of differences. First they can be triggered by an action – this action is usually the addition of a message to an Azure Message Storage table. Second, they run on an Azure Website (recently renamed to “Web Apps”) and as a result you can run a number of WebJobs entirely for free!

(Oh – if you don’t know what a Worker Processes is, they’re Azure’s answer to Scheduled Tasks. If you don’t know what a Scheduled Task is, you probably clicked the link to this blog post by mistake. Maybe you were looking for more info about the colour Azure, in which case – here you go – you’re welcome)

I’ve seen a couple of people I follow on Twitter talk about how awesome Azure WebJobs are recently. For example, Troy Hunt uses WebJobs for his pretty awesome and super-scaled “Have I Been Pwned” website:

And Karoline Klever recently discovered how awesome they are on one of her projects:

So I’ve been looking for the right opportunity to try these out more for myself and a few weeks ago I was able to!

As I said at the beginning of this post, WebJobs can be triggered by adding data to an Azure Message Storage table. This post will focus on creating a Message Storage table and adding a message to it. In part 2 of this “not yet an epic” blog post series I will show you how to create the actual WebJob and do something when your data is queued up in the Message Storage table.

First things first

If you haven’t used Azure before, you’re probably going to need to install the Azure SDK so you can access all of this goodness.

You may also need to install the Azure Storage NuGet package if it’s not installed already locally.

Set up a storage account

So if you haven’t used Azure Storage before, the first thin you will need to do is create a Storage account through the Azure Management Portal. Azure Storage is billed based on the size of what you’re storing, as well as the number of transactions in and out of the storage. The costs are pretty tiny so for a quick example such as this one you probably won’t even be charged! However if you’re feeling particularly tight-fisted or don’t have access to a credit card at the moment, you can use your local Azure emulator instead (this should have been installed along with the Azure SDK from the previous step).

If you do create a storage account in Azure, you will see from the Azure portal that your storage account has 3 services – Blobs, Tables and Queues. Blobs are used to store files, Tables are used for data storage, and Queues are used for storing messages. The service we will be using is the Queues service.

azure-storage-services

Before you write the code to add messages to the Queue storage you need to know two things – the connection string for your storage account and the name of the queue you’re adding messages to.

Connection String

To find the connection string for your storage account, view your storage account dashboard in the Azure Management portal, and click the ‘Manage Access Keys’ link at the bottom of the page. You should see a box that looks like this:

azure-access-keys

The Connection String for your Azure storage will looks like this:

DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Primary Access Key]

You can store this in your application in 2 ways – either add it as an AppSetting value in your web.config like this:

Or if you’re using an Azure Cloud Service for your site you can add it to the settings of the Cloud Service Role like this:

azure-role-connection-string

Message Queue Name

You can actually define this in your code, and if the queue with that name doesn’t exist, you can get your code to create it. We can see that in action when we look at the code in the next bit. WARNING – your message queue name needs to be all in lower case otherwise you (like me) will spend hours and hours trying to figure out why it isn’t working!!!

Don’t bore us, get to the chorus!

Ok so now we have the boring config stuff out of the way, the interesting code bit is refreshingly straight forward – you can add data to your Message Queue in just 6 lines of code!!!

// First, retrieve the storage account using your connection string – I’m using the Cloud Service Role settings in this example:
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));

// Next, create a queue client
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();

// Then retrieve a reference to a queue – here’s where you give your queue a name
CloudQueue queue = queueClient.GetQueueReference("mymediaqueue");

// Create the queue if it doesn’t already exist
queue.CreateIfNotExists();

// Penultimate step – create a message and add it to the queue.
CloudQueueMessage message = new CloudQueueMessage("this is the data I want to add to my message queue");

// Finally, add your message to the queue
queue.AddMessage(message);

That’s it – pretty simple eh?

In reality though, you probably want to store a message that is a bit more useful than the example above! The good news is that because the message queue stores a string value, you can store serialized objects in there, and they will be deserialized to objects by your WebJob code in the next step.

So you could do something like this:

var messageQueueData = new
{
  Id = 1,
  SiteUrl = "http://www.mysiteurl.com"
};

CloudQueueMessage message = new CloudQueueMessage(JsonConvert.SerializeObject(messageQueueData));

A quick final point to make – you can view the contents of your message queue at any time through Visual Studio’s Server Explorer. Just connect to your Microsoft Azure account and navigate through to your storage account. You will see the Queues service there and the queue that you just created. Double click on your queue and you should see the message you just added!

view-the-message-queue

And that’s all there is to it really – in part 2 I will show you how to create and debug a WebJob to process the data from your message queue. Exciting stuff!

Posted in Code, Uncategorized, Windows Azure Tagged with: , ,

AngularJS Workshop – Friday 10th April

angularjs logo featured image

I’m a lot more than slightly excited to announce that I will be running my first full-day AngularJS Fundamentals workshop in April! The workshop is happening on Friday 10th April in Bristol, and will be run in conjunction with Develop Me Training – an awesome tech training company that was set up last year to help bridge the technical skills gap in the South West region.

During the day I will be covering the following topics, as well as providing time to get stuck in to some code:

  • AngularJS Databinding
  • Controllers & Dependency Injection
  • Structuring your AngularJS application
  • Using AJAX
  • Single Page Applications
  • AngularJS Services
  • AngularJS Resources
  • AngularJS Promises

So if you’re based in the Bristol/Bath area and want to learn more about AngularJS fundamentals, then come along!

Follow this link for more details and to book

Posted in AngularJS, Uncategorized Tagged with:

Umbraco – The GetPreValues Method Using Datatype Name

umbraco-logo

PreValues are the data values stored in Umbraco datatypes, such as a Dropdown list or a Checkbox list. If you use such a datatype (e.g. a pre-populated dropdown list) in your Umbraco installation, and you’re retrieving your Umbraco data from the application cache then you will generally not need to worry (or even know!) about PreValues as the cache would serve up the selected value for you. However if you’re accessing your Umbraco data directly from the database (and yes I know that it’s not as quick as accessing the cache, but sometimes you just have to access the database!) you will need to understand how to get the PreValues and the corresponding data out of them.

Most of the examples you see on how to get PreValues and their corresponding data (including this one from our.umbraco.org) usually involve hard-coding the ID of the datatype like this:

int categoryDataTypeNodeID = 1210;

or like this:

XPathNodeIterator iterator = umbraco.library.GetPreValues(10611);

I prefer to use the datatype name rather than hard-coding the ID though, as the ID could change if you set up a new environment or a new database. So here’s how I get my PreValues.

First I get the datatype by name:

var dataType = Services.DataTypeService.GetDataTypeDefinitionByName("My Datatype Name");

Then I get the PreValues for that datatype using the GetPreValues() method as seen above, which returns an XPathNodeIterator object:

var preValuesFromDataType = library.GetPreValues(dataType.Id);

Now once you have this XPathNodeIterator object you can move through the iterator and do whatever you need to do with those values. In the following example I move through my PreValues and add them to a list of SelectListItem objects in a controller, so I can use them within a Select List control in a form:

preValuesFromDataType.MoveNext();

var myIterator = preValuesFromDataType.Current.SelectChildren(“preValue”, “”);

while (myIterator.MoveNext())
{
  model.Dropdown.Add(new SelectListItem { Text = myIterator.Current.Value, Value = myIterator.Current.GetAttribute("id", "") });
}

The important thing to note is that you need to call the MoveNext() method on your XPathNodeIterator (line 1 above) before you do the assignment statement (line 2 above), otherwise your “myIterator” object will be null and you won’t get anything out of it! But apart from that, it’s pretty straight forward once you know!

Posted in Code, Umbraco Tagged with: ,

About Maff

Maff Rigby

I'm a certified .Net, Umbraco and AngularJS freelance developer with over 15 years experience in the IT industry. As well as writing code I love to teach; I run a number of workshops and 1-1 coaching sessions on Angular JS and Umbraco, and share what I know and learn here!

I’m social (ish)

Connect with me on LinkedIn, follow me on Twitter, or fail to find me on Facebook.