Wednesday, December 18, 2019

Amazon Queus with Ballerina

Ballerina and it's Connectors

Ballerina is a general purpose programming language designed for system integration. Web Services and REST APIs can be integrated with Ballerina code. WSO2 Ballerina Integrator, supports many out of the box components, called Ballerina Connectors, to programatically integrate external services and APIs. For example, when you want to connect to a Salesforce API, you can import the Salesforce Connector module to the Ballerina code and call its methods to the relevant REST API provided by Salesforce.

Amazon Simple Queue Service (SQS)

Amazon SQS is a simple message queue API provided by Amazon. WSO2 Enterprise Integrator (WSO2 EI) provides Ballerina Amazon SQS Connector  to programatically interact with the REST API provided by Amazon SQS. Once you have created an account in Amazon SQS, you can get the credentials for SQS service, which can be given to the Ballerina Connector as a configuration file or as configuration parameters. Then you can perform queue creation, enqueue, dequeue and message-delete operations by calling the respective API methods provided by the Connector.

Using SQS Connector

In this blog I am going to discuss the simplest way to run a Ballerina code with Amazon SQS Connector in Ubuntu/Linux console to create a SQS queue and send a message into it. For more information on usage of Ballerina Integrator with SQS connector with VS Code Plugin please visit the relevant tutorial documentation.

Setting up Ballerina Environment


Go to the WSO2 Ballerina Integrator download page.
Download the WSO2 Ballerina Integrator with Download button and install it.
Check whether Ballerina is correctly installed in your machine by executing the following command.

$ ballerina -v

If the Ballerina integrator is correctly installed in your machine you will get the following output.

Ballerina 1.0.2
Language specification 2019R3

Start developing Ballerina Code


Go to the directory location you want to make the Ballerina code.

$ cd loc

Create a Ballerina file.

touch sqs.bal

Open the file with your preferred editor.

gedit sqs.bal &

Add the following content to start the coding.

import ballerina/log;
import wso2/amazonsqs;

public function main(string... args) {

}

Note how the Ballerina console logging module and Amazon SQS Connector module is imported into the code. The main function is the program entry point as many other languages. Anyway this code snippet will not yet build as these imports are not used in the code.

Defining SQS Configurations and Client


Now let's define the Amazon SQS Configuration object and the SQS Connector client object above the main method.

amazonsqs:Configuration configuration = {
    accessKey: "Access Key",
    secretKey: "Secret Access Key",
    region: "Region",
    accountNumber: ""
};

amazonsqs:Client sqsClient = new(configuration);

Replace the Access Key, Secret Access Key and the Region parameters with the credentials obtained from the Amazon SQS account creation stage. Then you can create a SQS queue manually and you can get the  parameter from the path of the queue path generated.

Create a Standard Queue in Amazon SQS


There are 2 types of queues defined in Amazon SQS, Standard and FIFO. In this example we are going to create a Amazon Standard Queue. In order to do that we invoke the sqsClient as follows by adding the following code in the main method.

string|error queueURL = sqsClient->createQueue("myNewQueue", {});

if (queueURL is string) {
    log:printInfo("Created queue URL: " + queueURL);
} else {
    log:printInfo("Error occurred while creating a queue");
}

If the queue creation process had encountered an error, the queueURL would become an error object and a string type otherwise. The string would be the queue URL as specified in the documentation

If the queue was created the queue URL would be printed something like https://sqs.us-east-2.amazonaws.com/613964236299/myNewQueue. Note the format of the URL.

https://<Region>.amazonaws.com/<Access_Key>/<Queue_Name>

Enqueue a Message to an SQS Queue


Once an SQS queue is created a message can be stored in the queue. Invoking the sendMessage in the connector would send a message into the queue. Note that the received queue context path /<Access_Key>/<Queue_Name> has to be used for accessing the queue.


amazonsqs:OutboundMessage|error response = sqsClient->sendMessage("Sample text message.", "/613964236299/myNewQueue", {});

if (response is amazonsqs:OutboundMessage) {
    log:printInfo("Sent message to SQS. MessageID: " + response.messageId);
}

If the above message sending got successful you would get a console output similar to the following.

Sent message to SQS. MessageID: 7e7511a4-68f6-4c94-98e7-2b1e30301a0b

The complete code used would look like following.

import ballerina/log;
import wso2/amazonsqs;

amazonsqs:Configuration configuration = {
    accessKey: "AKZAY3QCLPL7DE5YSNC3",
    secretKey: "r0RYhP0lputX6hiYvcB5VK7hiY+Id+rUI57b7Qjp",
    region: "us-east-2",
    accountNumber: "613964236299"
};

amazonsqs:Client sqsClient = new(configuration);

public function main(string... args) {
    string|error queueURL = sqsClient->createQueue("myNewQueue", {});

    if (queueURL is string) {
        log:printInfo("Created queue URL: " + queueURL);

amazonsqs:OutboundMessage|error response = sqsClient->sendMessage("Sample text message.", "/613964236299/myNewQueue", {});

if (response is amazonsqs:OutboundMessage) {
    log:printInfo("Sent message to SQS. MessageID: " + response.messageId);
}

    } else {
        log:printInfo("Error occurred while creating a queue");
    }

}



Friday, August 23, 2019

From ESB to Ballerina

Introduction to Conventional ESB

The main slogan for ESB was the ability to configure a process flow models in Enterprise mediation scenarios without using a programming language. As majority of integration requirement scenarios (e.g.: Content Based Routing, Message Transformation) can be modeled with a set of XML tags, it was expected to be used by lay people to write their own business logic without any programming skill. This positioning is common to almost all ESBs currently available in the market like Oracle ESB, IBM Integration Bus and Mulesoft ESB.

How it was like The Real Marriage with Conventional ESBs?


Though it was believed to be useful to configure a XML for mediation, the content to be written was verbose and lengthy to configure. Once a problem is detected in a XML configuration it took much time to fix, as the process is involved with reading documentation and needed some trial-and-error configurations to come up with the correct configuration. As more and more configurable parameters were added to each of the XML component, the readability and the configurability was reduced significantly with the maturity of the ESB. Then the question was why a simple programming code would not be able to replace the lengthy XML configurations.
Another trend came up with time was the Micro Service Architecture. As the services are to be spread across several lightweight containers, the conventional ESBs faced the challenge of deploying artifacts with less up-time and with a less memory footprint. With the maturity of ESBs the code base was large and was consist of multiple layers of architecture that added less value compared to the cost involved. It was difficult to fulfill the lightweight requirement of the Microservice world with XML based SOAP mediation where REST and JSON became the mainstream.

Birth of Ballerina

WSO2 was thinking hard how to address the above issues with existing ESBs. Some proposed solutions were to develop a Java API for integration scenarios, replacing the XML configurations. However they have found that the flexibility given by an API would not be sufficient for some scenarios. And the acquisition of Java by a commercial organization introduced some fear of its existence in future as an opensource language where an API would tightly couple the integration with Java language. Another aspect was the unavoidable performance implication due to its inherent garbage collection where the modern integration is supposed to be so faster than data processing in convention.
The decision was to develop a programming language focused on integration, supporting cloud native capabilities by nature. Inspired by the programming based paradigm used for integration by Apache Camel, Ballerina was born. First version of the Ballerina language is running on Java Virtual Machine (JVM) which is known as JBallerina. Ballerina syntax is converted to the Java byte code which is running on top of JVM. This addressed most of the issues related to unavailability of libraries for Ballerina language, as Java library APIs can be wrapped with Ballerina, while providing the expected development environment for integration developers. Anyway, the future of the Ballerina is not limited to the JVM dependency. In future the there will be a language called nBallerina which would run on a different libraries and runtimes supporting heterogeneous language libraries.

My Programming Experience with Ballerina


At the moment I am involved in developing a Ballerina based integrator for WSO2 Enterprise Integrator (WSO2 EI). It will be the next generation of WSO2 Enterprise Integration, facilitating users to develop integration scenarios with Ballerina language. As the first step we are developing a set of Ballerina Connectors which are analogous to WSO2 ESB Connectors. There we develop connectors with Ballerina language 1.0.0 alpha. I have seen the XML based mediation used in similar cases where the development with Ballerina is much intuitive relative to them. With Ballerina I could easily use my programming knowledge in Java, JavaScript and Python as a transferable skill. It was far easier than I expected except some issues I faced with the VSCode development environment. VSCode plugin for Ballerina is under heavy development at the moment which is expected to solve most of the usability issues in it. In conventional Integration platforms which use XML as the configuration language (e.g.: Mulesoft, IBM Integration) are highly prone to errors when it comes to error handling and incompatibility across different mediators/connectors as the issues are not interactively communicated to the developer. That highly increase the development time compared to developing a general purpose code. Ballerina has correctly addressed that deficiency by making the error handling mandatory by design and providing code completion suggestions and snippet generation for commonly used scenarios. As Ballerina supports messages to be defined as types, many run time errors caused due to mismatch of message structures are avoided at the compiler phase.

Final Wrap up

With my previous experience developing in ESBs based on XML configurations I feel easier to develop with Ballerina for similar scenarios. I tried my best to convey my opinion on Ballerina minimizing my biases on Ballerina. I expect the enterprise integration paradigm based on configuration would be moved to simple programming based models like Ballerina in future.