Hands-On Experience: Import Data to Nebula Graph With Spark

This article is written by Liu Jiahao, an engineer at the big data team of IntSig Information Co. Ltd (IntSig). He has been playing around with Nebula Graph and is one of our proud GitHub contributors. This post shares his experience importing data to Nebula Graph with Spark.

Why Nebula Graph?

The graph-related business has grown more and more complex, and performance bottlenecks are identified in some popular graph databases. For example, a single machine has difficulties in scaling to larger graphs. In terms of performance, the native graph storage of Neo4j has irreplaceable advantages. In my survey, JanusGraph, Dgraph, and other graph databases cannot be comparable to Neo4j in this regard. JanusGraph performs very well in OLAP and can support OLTP to some extent. However, this cannot be an advantage of JanusGraph anymore, because some technologies, such as GraphFrame, are sufficient for the OLAP requirements. Besides, since Spark 3.0 starts to support Cypher, I found that comparing with the OLTP requirements of graphs, their OLAP requirements can be satisfied with more technologies. Therefore, Nebula Graph undoubtedly turns out to be a breakthrough to the low efficiency distributed OLTP databases.

3 Best JavaScript Frameworks/Libraries for Front-End Development

JavaScript has grown in popularity over the years, the community is growing rapidly, and developers are continually developing and building tools for the language on a daily basis.

This makes it difficult to decide which tool/framework/library to use for a particular task, as there are always multiple options for literally anything you want to do in JavaScript. At first, it's always difficult to decide which library or framework to learn.

Data Modeling Tools Detailed Comparison

A data modeling tool or a database modeling tool is an application that helps data modelers to create and design databases structureThus, data modeling tools make the Data modeling process easier and provide many features that help data modelers to understand their data. 

Actuallythere are many different data modeling tools available for different database platforms. This multitude of tools available makes it very difficult to choose a tool that suits the user's needs.  

Machine Learning Youtube Videos for Data Scientists (Under 10 min.)

Machine learning educational content is often in the form of academic papers or blog articles. These resources are incredibly valuable. However, they can sometimes be lengthy and time-consuming. If you just want to learn basic concepts and don’t require all the math and theory behind them, concise machine learning videos may be a better option. The Youtube videos on this list cover concepts such as what machine learning is, the basics of natural language processing, how computer vision works, and machine learning in video games. 

1. What Is Machine Learning? | Machine Learning Basics


A Complete Guide To Develop A Cloud-Based Application

The term cloud computing is all the rage at present. Businesses from across the globe are more inclined towards cloud-based technology and are rapidly hosting in the cloud.

We can say that the cloud is a platform that hosts an abundance of computing resources over the internet as an easy-to-use and on-demand utility that can be utilized on a pay-as-you-go basis.

5 Reasons to Use GraphQL at Your Company

The Rise of GraphQL

What is the best way to build an API today? REST probably comes to mind, but if you’re going to make the investment to build new software, it’s probably worth considering a few different options and choosing the best among them.

GraphQL stands out as an alternative to the REST API architecture mainly (but not only) because it provides a discoverable API by design. It also comes with its own query language and a runtime for fulfilling queries via functions called resolvers.

KMeans Clustering implementation in Java

I came across a KMeans implementation in Java. The number of clusters generated by the code is 3. But when I tried to change it to 2, I am getting the error: Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 2

Any help is much appreciated. Here is the code:

public class KMeans 
{

    public static void main(String[] args) {

        int data[] = {2,4,10,12,3,20,30,11, 12, 14, 19, 21, 29, 31, 40};//,25,17,23};    // initial data

        int noofclusters = 2;

        int centroid[][]=new int[][]{
            {0,0,0},
            {2,4,30}
        };
        getCentroid(data,noofclusters,centroid);

    }//main

    public static int[][] getCentroid(int data[],int noofclusters,int centroid[][]){

        int distance[][]=new int[noofclusters][data.length];
        int cluster[]=new int[data.length];
        int clusternodecount[]=new int[noofclusters];

        centroid[0]=centroid[1];
        centroid[1]=new int[]{0,0,0};
        System.out.println("========== Starting to get new centroid =========");

        for(int i=0;i<noofclusters;i++){
            for(int j=0;j<data.length;j++){
                //System.out.println(distance[i][j]+"("+i+","+j+")="+data[j]+"("+j+")-"+centroid[0][i]+"="+(data[j]-centroid[0][i]));
                distance[i][j]=Math.abs(data[j]-centroid[0][i]);
                System.out.print(distance[i][j]+" ,");
                //System.out.println("Centroid: "+centroid[0][i]);
            }
            System.out.println();
        }

        for(int j = 0; j < data.length; j++){
            int smallerDistance=0;
            if(distance[0][j]<distance[1][j] && distance[0][j]<distance[2][j])
                smallerDistance=0;
            if(distance[1][j]<distance[0][j] && distance[1][j]<distance[2][j])
                smallerDistance=1;
            if(distance[2][j]<distance[0][j] && distance[2][j]<distance[1][j])
                smallerDistance=2;//

            centroid[1][smallerDistance]=centroid[1][smallerDistance]+data[j];
            clusternodecount[smallerDistance]=clusternodecount[smallerDistance]+1;
            cluster[j]=smallerDistance;

            //System.out.println("Centerid at 1:  "+centroid[1][smallerDistance]);
            //System.out.print(cluster[j]+", ");



        }
        //for(int j=0;j<data.length;j++)
        //System.out.println("c at out: "+cluster[j]);

                System.out.println("======================================== ");

                System.out.println("New clusters are ");
                // cluster[]= { 0  1   0  1  0  2  2  1}
                // data[]={2,4,-10,12,3,20,30,11};
                 for(int i=0;i<noofclusters;i++){               
                    System.out.print("C"+(i+1)+": ");
                     for(int l=0;l<data.length;l++){
                    if(cluster[l]==i)
                        System.out.print(data[l]+" ,");

                }
                System.out.println();
            }
                System.out.println("======================================== ");

        System.out.println("New centroid is ");

        for(int j=0;j<noofclusters;j++){
            centroid[1][j]=centroid[1][j]/clusternodecount[j];
            System.out.print(centroid[1][j]+",");
        }
        System.out.println();

        boolean isAchived=true;
        for(int j=0;j<noofclusters;j++){
            if(isAchived && centroid[0][j] == centroid[1][j]){
                isAchived=true;
                continue;
            }
            isAchived=false;
        }

        if(!isAchived){

            getCentroid(data,noofclusters,centroid);
        }

        if(isAchived){
            System.out.println("======================================== ");
            System.out.println(" Final Cluster is ");
            for(int i=0;i<noofclusters;i++){    
                              System.out.print("C"+(i+1)+":");
                for(int j=0;j<data.length;j++){
                    if(cluster[j]==i)
                        System.out.print(data[j]+" ,");

                }
                System.out.println();
            }
        }

        return centroid;

    }
}

Amazon DynamoDB Integration

This article helps any new developer who wants to connect to Amazon DynamoDB.

Prerequisites:

  • Developer should have working IDE
  • The developer should have an AWS account and have DynamoDb service added.
  • A sample DynamoDb Table to be created.

Configuration Needed:

In your class-path, you should have awsconfiguration.json created.  For android, these files should reside in the project folder -> res/raw folder.

Serverless in Financial Services

Introduction

The financial industry is going through a radical change; the new generation of customers expects more precise, immediate, and comprehensive services. The emerging fintech offerings completely reshape the industry. New regulations are constantly emerging and need to be applied timely. In 2020, the unexpected global pandemic is another accelerator for this revolution. Around the world branches and offices are closed, more services and operations have been moving online, on the cloud, and on devices. At the same time, in the world of technology, people are moving toward becoming cloud-native, more precisely Kubernetes- native. Which completely changes the mindset of deployment, packaging, software development, and even the structures of how the IT teams are formed. 

Financial institutions have shifted their focuses, and invest more heavily on the IT infrastructure in order to:

AOT Compilation Make Java More Power

I experimented with a previous article to explore about 6000 classes being loaded for a simple hello world spring boot rest application.  Although the Quarkus version seems optimized by reducing the number to 3300+, It is still way too much. In this article, I will introduce how we can use Java AOT compilation to eliminate the dead code in Java and therefore improve the performance dramatically.

The experiment uses Quarkus as a framework.

The Most Common Java Keytool Keystore Commands

Java Keytool Keystore Commands

The platform that manages the private keys and certificates is called Java Keytool. It has the ability through which public/private keys and certificate manage in addition to caching certificates. The storing place of keys and certificates is named by Java as Keystore. Java Keystore represents a file. The private keys are protected with a password in Keystore. The chain of trust and primary certificate trustworthiness is established by Keytool Keystore that is necessary to protect the private keys and certificates.

A unique alias is associated with each certificate in Java Keystore. First, you have to create a .jks file that will initially consist of only private keys. After that, CSR needs to be generated from which certificate will be generated. Then the certificate should be imported into the Keystore including root certificates. There are various functions that are performed by the Java Keytool like viewing of certificate details or a list of certificates consist of export a certificate.

Analyzing Functional in Rust

Being a developer, it is easy to jump on the hype train and learn or even use in pet projects the latest libraries, frameworks, and why not programming languages.

In a sector where everything evolves at a very high speed, sometimes it is difficult to keep the focus and not go crazy with the constant changes.

Using Docker Swarm Secrets to Store and Rotate your SSL Certificates with Nginx

What is Docker Swarm Secrets?


Docker Swarm has an excellent feature out of the box — Docker Swarm secrets. Using it, you can easily keep your sensitive data like credentials, TLS certificates, etc.

In terms of Docker Swarm services, a secret is a blob of data, such as a password, SSH private key, SSL certificate, or another piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or in your application’s source code. You can use Docker secrets to centrally manage this data and securely transmit it to only those containers that need access to it.

Mediator Pattern Using .NET

Introduction

Mediator pattern is a very well-known pattern in programming, and in this post, we will see what it is, what it brings to the table. A straightforward implementation to understand some of the situations where we can think of using it.

  • A mediator pattern encapsulates how objects interact and communicate with each other.
  • It promotes loose coupling of objects.
  • It facilitates many-to-many relationships/communication as well as one-to-many.
  • You can think of Mediator as a communication hub.

Examples

You can use a mediator pattern for many different use-cases. Following are a few of the requirements/situations where the Mediator pattern can help us greatly.

Implementing Locks in a Distributed Environment

As we know, locks are generally used to monitor and control access to shared resources by multiple threads simultaneously. They basically protect data integrity and atomicity in concurrent applications, i.e., only one thread at a time can acquire a lock on a shared resource, which otherwise is not accessible. But a lock in a distributed environment is more than just a mutex in a multi-threaded application. It is more complicated because the lock now has to be acquired across all the nodes, whereas any of the nodes in the cluster or the network can fail.

Here is the user story that we're going to consider to explain scenarios in the rest of this article. The application takes data in the user’s preferred format and converts it into a standardized format, like PDF, that can be uploaded to a government portal. There are two different micro-services of the application which do these things: Transformer and Rules Engine. We use Cassandra for persistence and Kafka as a message queue. Also, please note that the user request, once accepted, returns immediately. Once the PDF is generated, the user is notified about it asynchronously. This is achieved in a sequence of steps as follows:

7 Database Optimization Hacks for Web Developers

Optimizing your database comes with great rewards. Higher performance and increased query efficiency are just a few examples of these benefits.

However, the means aren’t always straightforward and may require changing the rules altogether within a developer team. Furthermore, the examples listed here might not work for your database, based on the system you use. In that case, try to follow the core principle and translate the action into the means your system allows.