Author Archives: icorda

Build a Watson IBM conversational bot in 15 minutes

Watson is a question answering system, also know as QA, developed by IBM (named after its IBM’s first CEO Thomas J. Watson) and capable to answer questions posed in natural language. Watson has a variety of applications and domains such as Healthcare , IoT, Education, Financial Services, Marking and Customer Engagement and recently Weather Forecasting. Having come across chatterbots (or chatbots) in the past and always being interested in the long lasting debate around Artificial Intelligence and Thinking Machines (the Turing Test, John Searle’s Chinese Room and many more), I wanted to try out the IBM Watson Conversational Bot and build a simple bot that can have an application in a customer service type of scenario.

Setting up a Free Account on Bluemix

It is possible to set up a free Bluemix account to experiment with Watson Conversation. Bluemix is the IBM Cloud Platform as a Service and signing up here you can build your bot by picking Conversation from the IBM Bluemix Catalog. The Lite Pricing Plan (Free) will cover all you need to get started with the bot, for example up to 5 workspaces, up to 25 intents and entities (we will clarify later in this entry what they are). It also includes 10.000 API calls per months (POST message method calls only).

After selecting the Lite Pricing Plan, a Conversation instance will be created and you will be presented with some info on Conversation and on the right side of the new window the option to launch the instance (via the “Launch tool” button).

Creating a Workspace

The very first step into this journey is to create a workspace where you project will reside. Each workspace corresponds to one and only one bot. The below screenshot shows a newly created Workspace named “My First Bot project”

Screen Shot 2017-10-28 at 20.06.57

Configuring Intents

An intent is a task/goal that the user wants to perform or achieve and is expressed by a particular user’s input . For example, changing their password, downgrading their price plan or making a complaint. When planning the intents in your bot application, it is important to consider what the customer/user would like to do in a specific scenario and, in turn, what the bot itself will be able to handle.

In our example, we will configure 4 classes of intents. 2 for opening and closing statements (greeting the customer’s adviser and parting at the end of conversation) and 2 that would cover requests on the customer’s account.

  • greetings
  • goodbyes
  • account
  • account_changes

This is the overall UI the user will be presented with showing “Intents”, “Entities” and “Dialog”:

Screen Shot 2017-10-28 at 20.08.51.png

Here are (some) intents for the class #greetings:

Screen Shot 2017-10-28 at 20.11.29.png

Here some of those for the class #goodbyes:

Screen Shot 2017-10-28 at 20.15.14.png

Now that we have covered opening and closing statements in a conversation, we can include some intents for the class #account:

Screen Shot 2017-10-28 at 21.00.29.png

 

Still we need to include some training example for out bot to handle the next step in our conversation/dialog. Here some example under the class #account_changes

Screen Shot 2017-10-28 at 21.02.32.png

 

There is a last step that in this example will pretty much cover anything else and nothing really specific to what normally a customer would like to ask, but still we need to catch other possible things that can be asked or said. This will be done under the class #anything_else

Screen Shot 2017-10-28 at 20.38.08.png

Configuring Entities

An entity is a term or an object as part of the customer’s input that is aimed at clarifying the intent behind it. When usually intents are verbs (close, pay, change), entities are nouns (an account, a bill, a plan). Ideally, you should group entities that are likely to trigger a similar response throughout your dialog. For example, @customerDetailsChanges, will include all elements in the customer’s address that might need amending or changing as well various operations on the account such as close, pay, change plan etc under the category @accountMaintenance:

Screen Shot 2017-10-28 at 20.47.25.png

Screen Shot 2017-10-28 at 20.50.54.png

Building and Testing the Dialog Flow

After specifying intents and entities, it is now the time to plan our dialog’s flow. From the Dialog Tab, we are going to set a very basic and initial dialog with the #greetings intent. We create a new node called “Greeting response” and let’s add a new response condition that, if matches the existing intents, it will be prompted to the customer.

On the right side of your workspace there is a speech bubble with three dots. When you click on it, it will open a chat window from where you can test the current built dialog. As you can see from the below screenshot, “Hi” is correctly recognised as part of the class of intents called #greetings and the response “Hi, how can I help?”.

Screen Shot 2017-10-28 at 22.59.30.png

 

Having started off with #greetings, now we need to build the rest of the dialog. We will need to add 3 additional nodes:

  • Account Management.
  • Account Change Requests
  • Goodbye response

#Account Management’s node targets the intent #account and the entity @accountMaintenance as set earlier in this entry.

Screen Shot 2017-10-29 at 20.52.47.png

 

Furthermore, we will need to create a node for the specific request the customer would like to enquiry about (e.g. change of address), as follows:

Screen Shot 2017-10-29 at 20.48.18.png

We also need to add another node called “Goodbye response” to wrap up the interaction with the customer:

Screen Shot 2017-10-29 at 20.37.54.png

Now, let’s test the dialog Greeting response -> Customer Details Changes -> Goodbye response, as follows:

Screen Shot 2017-10-29 at 20.46.40.png

Finally, let’s try out the bot that consists of Greeting response -> Account Management -> Goodbye response, as follows:

Screen Shot 2017-10-29 at 23.46.15

Next Steps

There are a few directions you can go from here . First of all, the above bot is a very basic one, so you can enrich it by planning some additional potential scenarios and including more potential responses and interactions. You can look at using Jump to actions, add variation to your responses or add response conditions (e.g. using context variables). You can also use the Conversation REST API to modify your dialog programmatically, meaning that you will not need to use the Conversation tool UI. Once you are satisfied with your bot, you can also deploy your application by connecting the workspace to an interface customers will use. For example, you can test your bot on Slack in just a few simple steps. Finally, you can use the Watson SDK to develop your own web application and then your using with your service or integrating with 3rd party services.

(Some) References:

IBM Watson official website

IBM Watson official YouTube channel

Watson API and documentation

Swagger Documentation for the Watson API

Watson Conversation 

Watson IoT GitHub repo

Watson Cognitive Cooking Community

Advertisements

Yahoo open-sources Vespa, its most important software release since Hadoop – SiliconANGLE

Source: Yahoo open-sources Vespa, its most important software release since Hadoop – SiliconANGLE

What is cool in Java 8 and what is cooler in Java 9!

JSLT: 5 minutes with core JSLT tags

JSLT is a tag library provided by Sun, which has support for common tasks such as iteration and conditional execution, database access, tags for manipulating XML documents, internationalization. It is also possible to integrate custom tags with the default JSTL tags. There are in total 6 JSLT tag library descriptors, a few from the core library are presented in this blog entry.

<c:if> tag

<code><c:if></code> identifies an if statement, as per the below example. It also includes <code><cl:out></code> displays the result of an expression:

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
    <title>IF conditional example</title>
  </head>
  <body>
     <c:set var="num" scope="session" value="${100}"/>    
        <c:if test="${num> 0}"/>
            <c:out value="${num}"/>
        </c:if>
  </body>
</html>

<c:choose> tag

The <code><c:choose></code> establishes the context for non mutually exclusive, meaning that there is room for multiple if else clauses via <code><c:when></code> and <code></c:otherwise></code> tags:

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
    <title>CHOOSE example</title>
  </head>
  <body>
    <c:set var="name" scope="session" value="aName"/> 
    <cl:choose>
      <cl:when test="${name=='aName'}">
         <cl:out value="${name}"></cl:out>
      </cl:when>
   <cl:otherwise>
      <cl:out value="not maching"></cl:out>
   </cl:otherwise>
</cl:choose>
</body>
</html>

<c:forEach> tag

The tag <code><c:forEach></code> is for the basic iteration functionality.

<html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
 <title>forEach iteration example</title>
 </head>
 <body>
    <c:forEach var="counter" begin="1" end="10">
       <c:out value="${counter}"/>
    </c:forEach>
 </body>
</html>

My Most Useful IntelliJ IDEA Keyboard Shortcuts | Roberto Cortez Java Blog

Source: My Most Useful IntelliJ IDEA Keyboard Shortcuts | Roberto Cortez Java Blog

GitHub Integration: why and how | Upsource Blog

GitHub integration is so easy with IntelliJ!

Source: GitHub Integration: why and how | Upsource Blog

5 minutes with Angular2 Google Maps

I usually work with Node.js and JS/Javascript and a while ago had some exposure to Ext JS (I can’t claim I know a great deal of it), so I was interested in finding out about AngularJS or React. After a full day Java development and some good fight with merge conflicts (I won in the end), I decided that AngularJS would be the “chosen one” (the Skywalker of the Javascript Web Application Frameworks). I followed the Angular CLI first to get started and build a basic application in TypeScript and took it from there by making changes as I go along with the API documentation. Once launched

ng serve --open

to serve my first application and having played a bit with app.component.html and app.component.css, I was eager to find out an example where Angular can shine and be amazing and I happily discovered Angular 2 Components for Google Maps aka angular2-google-maps. Here a few steps in order to create a Google Maps with two markers with a map info window attachef that displays on click, in a nutshell a very gentle “getting started” tutorial.

Let’s assume that Node.js and NPM (Node Package Manager) is already installed.

  • Install Typescript
npm install -g typescript@2.0.0
  • Install AngularJS CLI:
npm install -g angular-cli@webpack
  • Create your new project
ng new my-app-project
cd my-app-project

Install angular2-google-maps via NPM:

npm install angular2-google-maps --save

The most important files you will need to modify are inside

my-app-project/src/app

There you will find the following:

app.component.ts
app.module.ts
app.component.css
app.component.html

App.component.ts includes the building blocks of an Angular application: components. component is the combination of an HTML template and a component class that controls a portion of the screen.

import { Component } from '@angular/core';

@Component({
 selector: 'app-root',
 templateUrl: 'app.component.html',
 styleUrls: ['app.component.css'],
})
export class AppComponent {
 title: string = 'My first angular2-google-maps project';
 lat1: number = 51.531826;
 lng1: number = -0.124391;
 lat2: number = 51.511899;
 lng2: number = -0.123270;
}

In our case, the class controlling what it is displayed is called AppComponent and other than the title, it includes longitudine and latitude for two markers. It also refers to our html and css files.

App.module.ts, instead, describes how the elements of the application fit together. Every application has at the very least one Angular module, the root module that you bootstrap to launch the application for which the conventional name is AppModule. You will have a number of imports together with @NgModule, that is basically orchestrates how to run and compile the module code. It includes some main properties, as follows:

  • imports, that are additional modules to be supported, even custom-made;
  • providers used to make services and values known to the dependency injection framework;
  • declarations used to declare components, directives, pipes that belongs to the current module;
  • boostraps, that are the main components to be bootstrapped.
import { NgModule, ApplicationRef } from '@angular/core';
import { CommonModule } from '@angular/common';
import { FormsModule } from '@angular/forms';
import { AppComponent } from './app.component';

import { AgmCoreModule } from 'angular2-google-maps/core';

@NgModule({
 imports: [
 BrowserModule,
 CommonModule,
 FormsModule,
 AgmCoreModule.forRoot({
 apiKey: 'MY_API_KEY'
 })
 ],
 providers: [],
 declarations: [
 AppComponent
 ],
 bootstrap: [AppComponent]
})
export class AppModule {}

 Finally, we have the html and css for our newly created app:

<h1>{{ title }}</h1>

<sebm-google-map [latitude]="lat1" [longitude]="lng1" 
                 [latitude]="lat2" [longitude]="lng2">
 <sebm-google-map-marker [latitude]="lat1" [longitude]="lng1">
   <sebm-google-map-info-window [disableAutoPan]="true">
   This is the location of <strong>Kings Cross Station</strong>
   </sebm-google-map-info-window>
 </sebm-google-map-marker>
 <sebm-google-map-marker [latitude]="lat2" [longitude]="lng2">
   <sebm-google-map-info-window [disableAutoPan]="true">
   This is the location of <strong>Covent Garden</strong>
   </sebm-google-map-info-window>
 </sebm-google-map-marker>
</sebm-google-map>
.sebm-google-map-container {
 height: 300px;
 }

The result would then look like this:

 

Screen Shot 2017-05-21 at 20.17.05.png

APIs for Spark Development: Java vs Scala

  • Apache Spark is an open source cluster computing platform for data processing tasks
  • It extends Apache Hadoop and introduces concepts such as stream processing (Spark Streaming) and Iterative Computation (Machine Learning tasks)
  • Apache Spark was initially written in Scala and ships with a Scala and Python interactive shell – REPL(Read-Evaluate-Print-Loop). It includes the following APIs:
    • Java
    • Scala
    • Python

Spark APIs: Java Vs Scala

Java

  • Java less concise and more verbose and error prone – support for lambdas and stream only from Java 8
  • More Established programming language, lots of experts in the market
  • Full Java/Scala interoperability – implicit conversions between major Collections’ types

Scala

  • Blend of functional and object-oriented aspects makes Scala highly scalable
  • No distinction between an object and a function – every value is an object and every operation is a method call
  • Scala’s type inference contributes to more readable programs
  • Scala’s Traits tames multiple inheritance
  • Scala displays conciseness, brevity and advanced static typing.

Scala API at work: Loading CSV Files

ScalaApiExample

Java Api at work: Loading CSV Files

Java

Final Remarks

  • Scala’s strengths lay on Scalability, Conciseness and Advanced Static Typing together with full Java interoperability
  • Scala can have a challenging learning curve and still has limited community presence (compared to Java and Python)
  • Scala developers are still a niche Vs Rich Market for Java and Python professionals
  • Python still a strong choice because of easy transition from OOP languages and the number of available statistical and Data Science libraries

Read a MongoDB Collection from Spark: an example

I was recently presented with the task of reading an existing MongoDb collection from Spark, querying its content via Spark-SQL and export it as a csv. I have ended up using Spark-MongoDB library, which allows reading and writing data with Spark-SQL from or into MongoDB. The collection in question is the one offered for free by MongDB here and the jars used in my pom.xml are as follows:

 

</pre>
<pre>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
    &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;

    &lt;groupId&gt;mongodb-spark&lt;/groupId&gt;
    &lt;artifactId&gt;mongodb-spark&lt;/artifactId&gt;
    &lt;version&gt;1.0-SNAPSHOT&lt;/version&gt;
    &lt;dependencies&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
            &lt;artifactId&gt;spark-core_2.10&lt;/artifactId&gt;
            &lt;version&gt;1.6.1&lt;/version&gt;
            &lt;scope&gt;compile&lt;/scope&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;junit&lt;/groupId&gt;
            &lt;artifactId&gt;junit&lt;/artifactId&gt;
            &lt;version&gt;4.12&lt;/version&gt;
            &lt;scope&gt;test&lt;/scope&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;com.stratio.datasource&lt;/groupId&gt;
            &lt;artifactId&gt;spark-mongodb_2.10&lt;/artifactId&gt;
            &lt;version&gt;0.11.2&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;mongodb-driver&lt;/artifactId&gt;
            &lt;version&gt;3.2.2&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;mongo-java-driver&lt;/artifactId&gt;
            &lt;version&gt;3.2.2&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
            &lt;artifactId&gt;spark-sql_2.10&lt;/artifactId&gt;
            &lt;version&gt;1.6.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;casbah-commons_2.10&lt;/artifactId&gt;
            &lt;version&gt;3.1.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;casbah-core_2.10&lt;/artifactId&gt;
            &lt;version&gt;3.1.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;casbah-query_2.10&lt;/artifactId&gt;
            &lt;version&gt;3.1.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;com.databricks&lt;/groupId&gt;
            &lt;artifactId&gt;spark-csv_2.10&lt;/artifactId&gt;
            &lt;version&gt;1.4.0&lt;/version&gt;
        &lt;/dependency&gt;
    &lt;/dependencies&gt;
&lt;/project&gt;</pre>
<pre>

Here is my Java class:

</pre>
<pre>import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;

import java.util.HashMap;
import java.util.Map;

public class MongoDBSparkIntegration {

    public static void main(String[] args) {

        String outputPath = "D:\\Dev\\MongoDb-Spark-Integration\\src\\main\\resources\\output";

        SparkConf sparkConf = new SparkConf().setMaster("local").setAppName("MongoDB-Spark-Application");
        JavaSparkContext sc = new JavaSparkContext(sparkConf);
        SQLContext sqlContext = new org.apache.spark.sql.SQLContext(sc);
        Map&lt;String, String&gt; options = new HashMap&lt;String, String&gt;();
        options.put("host", "localhost:27017");
        options.put("database", "mydb");
        options.put("collection", "test");

        DataFrame df = sqlContext.read().format("com.stratio.datasource.mongodb").options(options).load();
        df.registerTempTable("tmp");
        sqlContext.sql("SELECT * FROM tmp").show();
        df.write().format("com.databricks.spark.csv").save(outputPath);
        sqlContext.dropTempTable("tmp");
    }
}</pre>
<pre>

This is the printout returned from the IntelliJ console:

Capture

 

Java 8 vs. Scala: Part I – DZone Performance

Source: Java 8 vs. Scala: Part I – DZone Performance

%d bloggers like this: