Yahoo open-sources Vespa, its most important software release since Hadoop – SiliconANGLE

Source: Yahoo open-sources Vespa, its most important software release since Hadoop – SiliconANGLE

Advertisements

What is cool in Java 8 and what is cooler in Java 9!

JSLT: 5 minutes with core JSLT tags

JSLT is a tag library provided by Sun, which has support for common tasks such as iteration and conditional execution, database access, tags for manipulating XML documents, internationalization. It is also possible to integrate custom tags with the default JSTL tags. There are in total 6 JSLT tag library descriptors, a few from the core library are presented in this blog entry.

<c:if> tag

<code><c:if></code> identifies an if statement, as per the below example. It also includes <code><cl:out></code> displays the result of an expression:

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
    <title>IF conditional example</title>
  </head>
  <body>
     <c:set var="num" scope="session" value="${100}"/>    
        <c:if test="${num> 0}"/>
            <c:out value="${num}"/>
        </c:if>
  </body>
</html>

<c:choose> tag

The <code><c:choose></code> establishes the context for non mutually exclusive, meaning that there is room for multiple if else clauses via <code><c:when></code> and <code></c:otherwise></code> tags:

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
    <title>CHOOSE example</title>
  </head>
  <body>
    <c:set var="name" scope="session" value="aName"/> 
    <cl:choose>
      <cl:when test="${name=='aName'}">
         <cl:out value="${name}"></cl:out>
      </cl:when>
   <cl:otherwise>
      <cl:out value="not maching"></cl:out>
   </cl:otherwise>
</cl:choose>
</body>
</html>

<c:forEach> tag

The tag <code><c:forEach></code> is for the basic iteration functionality.

<html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
 <title>forEach iteration example</title>
 </head>
 <body>
    <c:forEach var="counter" begin="1" end="10">
       <c:out value="${counter}"/>
    </c:forEach>
 </body>
</html>

My Most Useful IntelliJ IDEA Keyboard Shortcuts | Roberto Cortez Java Blog

Source: My Most Useful IntelliJ IDEA Keyboard Shortcuts | Roberto Cortez Java Blog

GitHub Integration: why and how | Upsource Blog

GitHub integration is so easy with IntelliJ!

Source: GitHub Integration: why and how | Upsource Blog

5 minutes with Angular2 Google Maps

I usually work with Node.js and JS/Javascript and a while ago had some exposure to Ext JS (I can’t claim I know a great deal of it), so I was interested in finding out about AngularJS or React. After a full day Java development and some good fight with merge conflicts (I won in the end), I decided that AngularJS would be the “chosen one” (the Skywalker of the Javascript Web Application Frameworks). I followed the Angular CLI first to get started and build a basic application in TypeScript and took it from there by making changes as I go along with the API documentation. Once launched

ng serve --open

to serve my first application and having played a bit with app.component.html and app.component.css, I was eager to find out an example where Angular can shine and be amazing and I happily discovered Angular 2 Components for Google Maps aka angular2-google-maps. Here a few steps in order to create a Google Maps with two markers with a map info window attachef that displays on click, in a nutshell a very gentle “getting started” tutorial.

Let’s assume that Node.js and NPM (Node Package Manager) is already installed.

  • Install Typescript
npm install -g typescript@2.0.0
  • Install AngularJS CLI:
npm install -g angular-cli@webpack
  • Create your new project
ng new my-app-project
cd my-app-project

Install angular2-google-maps via NPM:

npm install angular2-google-maps --save

The most important files you will need to modify are inside

my-app-project/src/app

There you will find the following:

app.component.ts
app.module.ts
app.component.css
app.component.html

App.component.ts includes the building blocks of an Angular application: components. component is the combination of an HTML template and a component class that controls a portion of the screen.

import { Component } from '@angular/core';

@Component({
 selector: 'app-root',
 templateUrl: 'app.component.html',
 styleUrls: ['app.component.css'],
})
export class AppComponent {
 title: string = 'My first angular2-google-maps project';
 lat1: number = 51.531826;
 lng1: number = -0.124391;
 lat2: number = 51.511899;
 lng2: number = -0.123270;
}

In our case, the class controlling what it is displayed is called AppComponent and other than the title, it includes longitudine and latitude for two markers. It also refers to our html and css files.

App.module.ts, instead, describes how the elements of the application fit together. Every application has at the very least one Angular module, the root module that you bootstrap to launch the application for which the conventional name is AppModule. You will have a number of imports together with @NgModule, that is basically orchestrates how to run and compile the module code. It includes some main properties, as follows:

  • imports, that are additional modules to be supported, even custom-made;
  • providers used to make services and values known to the dependency injection framework;
  • declarations used to declare components, directives, pipes that belongs to the current module;
  • boostraps, that are the main components to be bootstrapped.
import { NgModule, ApplicationRef } from '@angular/core';
import { CommonModule } from '@angular/common';
import { FormsModule } from '@angular/forms';
import { AppComponent } from './app.component';

import { AgmCoreModule } from 'angular2-google-maps/core';

@NgModule({
 imports: [
 BrowserModule,
 CommonModule,
 FormsModule,
 AgmCoreModule.forRoot({
 apiKey: 'MY_API_KEY'
 })
 ],
 providers: [],
 declarations: [
 AppComponent
 ],
 bootstrap: [AppComponent]
})
export class AppModule {}

 Finally, we have the html and css for our newly created app:

<h1>{{ title }}</h1>

<sebm-google-map [latitude]="lat1" [longitude]="lng1" 
                 [latitude]="lat2" [longitude]="lng2">
 <sebm-google-map-marker [latitude]="lat1" [longitude]="lng1">
   <sebm-google-map-info-window [disableAutoPan]="true">
   This is the location of <strong>Kings Cross Station</strong>
   </sebm-google-map-info-window>
 </sebm-google-map-marker>
 <sebm-google-map-marker [latitude]="lat2" [longitude]="lng2">
   <sebm-google-map-info-window [disableAutoPan]="true">
   This is the location of <strong>Covent Garden</strong>
   </sebm-google-map-info-window>
 </sebm-google-map-marker>
</sebm-google-map>
.sebm-google-map-container {
 height: 300px;
 }

The result would then look like this:

 

Screen Shot 2017-05-21 at 20.17.05.png

APIs for Spark Development: Java vs Scala

  • Apache Spark is an open source cluster computing platform for data processing tasks
  • It extends Apache Hadoop and introduces concepts such as stream processing (Spark Streaming) and Iterative Computation (Machine Learning tasks)
  • Apache Spark was initially written in Scala and ships with a Scala and Python interactive shell – REPL(Read-Evaluate-Print-Loop). It includes the following APIs:
    • Java
    • Scala
    • Python

Spark APIs: Java Vs Scala

Java

  • Java less concise and more verbose and error prone – support for lambdas and stream only from Java 8
  • More Established programming language, lots of experts in the market
  • Full Java/Scala interoperability – implicit conversions between major Collections’ types

Scala

  • Blend of functional and object-oriented aspects makes Scala highly scalable
  • No distinction between an object and a function – every value is an object and every operation is a method call
  • Scala’s type inference contributes to more readable programs
  • Scala’s Traits tames multiple inheritance
  • Scala displays conciseness, brevity and advanced static typing.

Scala API at work: Loading CSV Files

ScalaApiExample

Java Api at work: Loading CSV Files

Java

Final Remarks

  • Scala’s strengths lay on Scalability, Conciseness and Advanced Static Typing together with full Java interoperability
  • Scala can have a challenging learning curve and still has limited community presence (compared to Java and Python)
  • Scala developers are still a niche Vs Rich Market for Java and Python professionals
  • Python still a strong choice because of easy transition from OOP languages and the number of available statistical and Data Science libraries

Read a MongoDB Collection from Spark: an example

I was recently presented with the task of reading an existing MongoDb collection from Spark, querying its content via Spark-SQL and export it as a csv. I have ended up using Spark-MongoDB library, which allows reading and writing data with Spark-SQL from or into MongoDB. The collection in question is the one offered for free by MongDB here and the jars used in my pom.xml are as follows:

 

</pre>
<pre>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
    &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;

    &lt;groupId&gt;mongodb-spark&lt;/groupId&gt;
    &lt;artifactId&gt;mongodb-spark&lt;/artifactId&gt;
    &lt;version&gt;1.0-SNAPSHOT&lt;/version&gt;
    &lt;dependencies&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
            &lt;artifactId&gt;spark-core_2.10&lt;/artifactId&gt;
            &lt;version&gt;1.6.1&lt;/version&gt;
            &lt;scope&gt;compile&lt;/scope&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;junit&lt;/groupId&gt;
            &lt;artifactId&gt;junit&lt;/artifactId&gt;
            &lt;version&gt;4.12&lt;/version&gt;
            &lt;scope&gt;test&lt;/scope&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;com.stratio.datasource&lt;/groupId&gt;
            &lt;artifactId&gt;spark-mongodb_2.10&lt;/artifactId&gt;
            &lt;version&gt;0.11.2&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;mongodb-driver&lt;/artifactId&gt;
            &lt;version&gt;3.2.2&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;mongo-java-driver&lt;/artifactId&gt;
            &lt;version&gt;3.2.2&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
            &lt;artifactId&gt;spark-sql_2.10&lt;/artifactId&gt;
            &lt;version&gt;1.6.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;casbah-commons_2.10&lt;/artifactId&gt;
            &lt;version&gt;3.1.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;casbah-core_2.10&lt;/artifactId&gt;
            &lt;version&gt;3.1.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;org.mongodb&lt;/groupId&gt;
            &lt;artifactId&gt;casbah-query_2.10&lt;/artifactId&gt;
            &lt;version&gt;3.1.1&lt;/version&gt;
        &lt;/dependency&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;com.databricks&lt;/groupId&gt;
            &lt;artifactId&gt;spark-csv_2.10&lt;/artifactId&gt;
            &lt;version&gt;1.4.0&lt;/version&gt;
        &lt;/dependency&gt;
    &lt;/dependencies&gt;
&lt;/project&gt;</pre>
<pre>

Here is my Java class:

</pre>
<pre>import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;

import java.util.HashMap;
import java.util.Map;

public class MongoDBSparkIntegration {

    public static void main(String[] args) {

        String outputPath = "D:\\Dev\\MongoDb-Spark-Integration\\src\\main\\resources\\output";

        SparkConf sparkConf = new SparkConf().setMaster("local").setAppName("MongoDB-Spark-Application");
        JavaSparkContext sc = new JavaSparkContext(sparkConf);
        SQLContext sqlContext = new org.apache.spark.sql.SQLContext(sc);
        Map&lt;String, String&gt; options = new HashMap&lt;String, String&gt;();
        options.put("host", "localhost:27017");
        options.put("database", "mydb");
        options.put("collection", "test");

        DataFrame df = sqlContext.read().format("com.stratio.datasource.mongodb").options(options).load();
        df.registerTempTable("tmp");
        sqlContext.sql("SELECT * FROM tmp").show();
        df.write().format("com.databricks.spark.csv").save(outputPath);
        sqlContext.dropTempTable("tmp");
    }
}</pre>
<pre>

This is the printout returned from the IntelliJ console:

Capture

 

Java 8 vs. Scala: Part I – DZone Performance

Source: Java 8 vs. Scala: Part I – DZone Performance

How to learn Scala – Codacy | Blog

A list of resources to getting started with Scala. Become a (better) Scala developer with a list of resources, tutorials and best practices.

Source: How to learn Scala – Codacy | Blog

%d bloggers like this: