For the past week Ive been obsessed with Node-RED. If you are not familiar with it, Node-RED is an open source IBM technology for wiring together hardware devices, APIs and online services in new and interesting ways. Since Node-RED is built on Node.js, this makes it ideal to run at the edge of the network on low-cost hardware such as an Arduino or Raspberry Pi as well as in the cloud.
Similar in methodology to Yahoo! Pipes, Node-RED initially started as a visual tool for wiring IoT but has developed into a service that can do all sorts of crazy things. Installation is simple but to make life easier, IBM Bluemix offers a Node-RED template so you can get a web app up and running in no time.
I went through the create your first flows tutorials but I wanted to make something more substantial utilizing some of the super sweet IBM Watson services. Heres what I created, and in the end, it was ridiculously simple to build.
Here's an overview of the Node-RED flow:
- Calls the topcoder RSS feed for JavaScript challenges every hour.
- For each new challenge, call the challenge API to get its details.
- The flow only processes challenges where the type is 'Assembly Competition and the total prize money is greater than $1500.
- Translate the name of the challenges and its requirements into Spanish using the Watson Machine Translation service.
- Save the Spanish overview of the challenge to MongoDB.
- Tweet the challenge name (in Spanish) and a link to this application so that the Spanish requirements can be viewed.
The flow tweets a link to the app (http://jeffdonthemic-node-red.mybluemix.net/challenge?id=:id) so that people can view the Spanish version of the challenge. Building the REST endpoint was almost laughable. Node-RED provides input and output node for http request/response, allowing for the creation of simple web services. My endpoint does the following:
- Fetches the challenge from MongoDB using the challenge id passed.
- Generates output HTML with the Spanish data using a mustache template.
- Writes the template to the output stream displaying the Spanish challenge content in the browser.
IBM Bluemix Setup
Login to your IBM Bluemix console and create a new Cloud Foundry App using the Node-RED Starter boilerplate. Ensure you add a MongoDB service (I chose MongoLab) and the Watson Language Translation service. Youll need the credentials from the Language Translation service when building the flow so click the Show Credentials link and keep them handy.
Building the Flow
Our application is simply built by finding a node in the left panel, dragging it onto the tab and connecting them together.
To get started, add a Feedparse node for the topcoder RSS feed and enter this URL for the feed URL.
The feedparse node passes a msg.topic for each entry so parse the ID of the challenge and build the msg.url to pass to the next node.
Make a GET request to the url passed in msg.url (instead of in the URL form field) and pass the response to the next node as a parsed JSON object.
Add a Switch node so that we only process flows where the challenge type is 'Assembly Competition.
Add a Function node that calculates the total prize amount and passes it along in the message.
Now we add another Switch node which continues the flow only if the total prize money is greater than $1500.
Add another all purpose Function node and prep the challenge data for translation. Add the challenge object to the payload and challenge name itself to the payload for the next node (Watson Translation).
Add the Watson Translation node and enter your credentials for the service. The node translates the name and passes it to the next node as its payload.
Add another function node and set the challenge name to the newly translated challenge name and add the requirement payload so that they can be translated.
Add another Watson Translation node and translate the requirements into Spanish.
Add a final Function node and prep the payload to be saved in MongoDB and tweeted. First we replace the English requirements with the newly translated Spanish ones and create two object for the outputs. We then wire these two outputs to their corresponding nodes.
Add a mongo output node to save the msg.payload object to the challenges collection. Connect the top output in the previous node to this one.
Lastly, add a twitter out node, connect the bottom output from the previous node to it and then authenticate with your twitter credentials.
If you deploy the workspace now it should run the Feedparse and automagically persist data to MongoDB and tweet assuming everything is setup properly.
Building the Web Service
Now we need to build our web service so that when people click on the link in the tweet, it displays the challenge information in Spanish in their browser.
Add a new sheet to your workspace, drag an http input node onto it and setup the method and URL. When the /challenges endpoint receives a request it passes it to the next node.
Add a Function node that simply parses the challenge id from the URL string and adds it to the payload.
Then use a MongoDB node to find the challenge record by id in the challenges collection.
Add another Function node that simply uses the first record returned in the array from MongoDB and adds it to the msg.payload. This is done to make it easier to work with in the templating node.
Lastly, we want to make the HTML for the user (somewhat) pretty so we use a Template node and use mustache to format the msg.payload.
Then we simply use an http response node to send the HTML response back to the initial http request and we are done!
Results
Now when the flow receives new challenge entries in the RSS feed, it will run the flow then tweet the challenge info in Spanish and display the Spanish version of the challenge on the web page for users to view.
