Posted on

The Ultimate Guide to Data Collection in Data Science

In today’s world, data plays a key role in the success of any business. Data produced by your target audience, your competitors, information from the field you work and data your company gains on its own may help you find more customers, analyze your business decisions, reoptimize the business model or escalate to other markets. Data will help you define problems your business can solve and provide better service, specifying precisely your clients' needs. 

According to The McKinsey Global Institute research, data-driven companies are 23 times more likely to acquire customers, six times as likely to retain customers, and 19 times as likely to be profitable.  

Posted on

How To Ensure Data Transparency and Why It’s Important

Data transparency is more important than ever before, with more people going online amid surging cyberthreats. Consumers are becoming more aware and critical of how software and websites use their data. More than a few companies have gotten bad publicity due to a lack of transparency about user data. 

How can developers ensure they are providing good data transparency? A few key tactics will help.

Posted on

How to Prepare for a Personal Data Compliance Audit

As the basis for the main requirements for data protection, we will consider the EU GDPR as the most pervasive and influential legislation in this area. In this article, we will skip the legal and organizational parts of the regulation which you can read elsewhere, and jump right in to explain what technical measures you can implement to get compliant.

If you have a compliance check scheduled you will need to have the following in place:

Posted on

Benefits of Data Ingestion

Introduction

In the last two decades, many businesses have had to change their models as business operations continue to complicate. The major challenge companies face today is that a large amount of data is generated from multiple data sources. So, data analytics have introduced filters to various data sources to detect this problem. They need analytics and business intelligence to access all their data sources to make better business decisions.

It is obvious that the company needs this data to make decisions based on predicted market trends, market forecasts, customer requirements, future needs, etc. But how do you get all your company data in one place to make a proper decision? Data ingestion consolidates your data and stores it in one place.

Posted on

Data Collection Techniques for Market Research

Market research is critical for every type of business. It gives you insights into your target audience and how you should present your products. 

Unfortunately, 95% of new products fail. However, with the right data collection techniques, you can make your product unique and better than what is already present in the market. 

Posted on

Why MQTT Is Essential for Building Connected Cars

The automotive industry is embracing the idea of building a connected car. They see opportunity in using telemetry data from vehicles to create new revenue opportunities, and to build a better user experience. However, implementing a connected car service that can scale to support millions of cars can present some challenges.

For most connected car services, there is a requirement for bi-directional communication between the car and the cloud. Cars will send telemetry data to the cloud and enable apps like predictive maintenance, assisted driving, etc. Similarly, the car needs to be able to receive messages from the cloud to respond to remote commands, like remote lock/unlock door and remote activation of horn or lights.

Posted on

Preview and Snapshot Features in StreamSets Data Collector

Hello from your newly-appointed community champion and technical evangelist here at StreamSets! My name is Dash Desai and you will find me writing blog posts and cruising the community forums answering questions about StreamSets Data Collector as well as learning from community members. I will also be presenting at meetups and conferences so if you happen to be attending, please stop by and say hi. My first post for StreamSets, explaining the powerful Preview and Snapshot features in Data Collector, was inspired by one of the community members (Thank you, Edward).

Introduction

When creating data pipelines for big data projects and working with a diverse set of structured, semi-structured, and unstructured data sources, it is imperative that you get a true sense of the data transformations at every stage. Not just to ensure data integrity and data quality but also for debugging and audit trail purposes. So phrases like "Garbage in, Garbage out", " Fail fast, Fail often", and " Agile and Iterative development " are also applicable to creating dataflow pipelines.

Posted on

Building an Open Source Mixpanel Alternative, Part 1: Collecting and Displaying Events

This is the first part of a tutorial series on building an analytical web application with Cube.js. It expects the reader to be familiar with Javascript, Node.js, React, and have basic knowledge of SQL. The final source code is available here and the live demo is here. The example app is serverless and running on AWS Lambda. It displays data about its own usage.

There is a category of analytics tools like Mixpanel or Amplitude, which are good at working with events data. They are ideal for measuring product or engagement metrics, such as activation funnels or retention. They are also very useful for measuring A/B tests.