Author: Miguel Garcia

In this article, we are going to look at the challenges faced when rapidly scaling engineering teams in startup companies as well as other kinds of companies with a focus on product development. These challenges change between different types of companies, sizes, and stages of maturity. For instance, the growth of a consultancy software company focused on outsourcing is so different from a startup focused on product development.

I've faced much team growth and also seen the growth of teams in several companies, and most of them have faced the same challenges and problems.

This is an article from DZone's 2023 Data Pipelines Trend Report.

For more:

Read the Report

Originally, the term "data pipeline" was focused primarily on the movement of data from one point to another, like a technical mechanism to ensure data flows from transactional databases to destinations such as data warehouses or to aggregate this data for analysis. Fast forward to the present day, data pipelines are no longer seen as IT operations but as a core component of a business's transformation model. 

In this article, we will explain how using the new Generative AI Models (LLM) can improve the experience of business users on our analytical platform. Let's say we provide our retail merchandising managers with a web application or a mobile application where they can analyze sales and stock behavior in real-time using natural language.

These applications usually have a series of restrictions that mainly show a generic type of analysis, which users can filter or segment based on some filters and provide information such as:

In this article, we are going to analyze a new data orchestrator: Dagster. In our opinion, this is the first generation of data orchestrators that bring data pipelines closer to critical business processes that would really be business data processes for mission-critical solutions. To describe Dagster's capabilities and use cases, we are going to provide some context about patterns and some historical information that is necessary to understand what business value it brings.

In the last decade, many trends have been around orchestration and choreography patterns. We are going to provide a simple description of these patterns:

This article is a follow-up to Engineering Manager: Resolving Intrapersonal Conflicts and Engineering Manager: Resolving Interpersonal Situations. In the previous articles, we talked about interpersonal conflicts, which are conflicts between two or more people that do not agree on topics such as what actions to take or what the priorities are. Today we will talk about organizational conflicts, common scenarios, and some tips on how to manage them in a professional environment.

Organizational

This conflict is associated with misalignment or disagreement between members of the organization and the organization's cultural values, strategic decisions, goals, or methodologies. Usually, this kind of conflict starts when people believe that their values, professional development, goals, or attitude are not aligned with the organization or with the management layer of their areas.

This article is a follow-up to the Engineering Manager: Resolving Conflict Situations, Part 1. In the previous article, we talked about intra-personal conflicts which are associated with people themselves. Today we will talk about interpersonal conflicts, common scenarios, and some tips on how to manage them in a professional environment.

Interpersonal

Interpersonal conflict is between two or more people that do not agree on topics such as what actions to take, what the priorities are, or how to resolve a situation. This kind of conflict is the most frequent and is usually the easiest to solve, but if poorly managed can generate an atmosphere of resentment between people.

In this article, a follow-up to the engineering manager series, we will talk about conflicts, common scenarios, and some tips on how to manage them in a professional environment. Conflict management is a complex topic and we will address it in a series of three articles, the first of which is this one.

People's conflicts are common in the professional day-to-day, some people might even say that conflict is not professional behavior, but of course, it is, as we are human so conflicts are inevitable. What makes the difference is how we manage these situations. 

This is an article from DZone's 2022 Data Pipelines Trend Report.

For more:

Read the Report

The current challenge in the world of data is no longer in the processing capacity of volumes of data. Because of the performance of modern current streaming platforms and owing to a new generation of data repositories that allow decoupling computation from the storage layer, we can improve scalability with very low operational effort.