Lior Gavish

Engineering at Monte Carlo

Joined Apr 2022

http://www.montecarlodata.com

About

CTO and Co-founder, Monte Carlo. Programming wizard and lover of cats.

Stats

Reputation:	722
Pageviews:	89.3K
Articles:	22
Comments:	1

Articles
Comments

Articles

Building Ethical AI Starts With the Data Team, Here’s Why

GenAI is an ethical quagmire. What responsibility do data leaders have to navigate it? We consider the need for ethical AI and why data ethics are AI ethics.

March 26, 2024

· 814 Views · 1 Like

When a Data Mesh Doesn’t Make Sense for Your Organization

Data mesh—including data mesh governance—requires the right mix of process, tooling, and internal resources. Find out what it takes to get data mesh-ready.

March 11, 2024

· 1,674 Views · 1 Like

5 Hard Truths About Generative AI for Technology Leaders

Creating GenAI that delivers value requires effort. But it's worth it. Learn how to bring your distinctive GenAI project to fruition.

January 11, 2024

· 3,718 Views · 3 Likes

Data Observability: Reliability in the AI Era

For GenAI, data observability must prioritize resolution, pipeline efficiency, and streaming and vector infrastructures.

December 2, 2023

· 2,951 Views · 2 Likes

The Moat for Enterprise AI Is RAG + Fine Tuning: Here’s Why

Generative AI is top of mind for nearly every CEO, but how it achieves enterprise readiness is up for debate. Here’s why RAG and fine tuning are the keys to its success.

November 17, 2023

· 2,346 Views · 1 Like

Generative AI and the Future of Data Engineering

Generative AI is revolutionizing the world. Proper data engineering and observability are critical for its success. Here's what it means for data engineering.

July 12, 2023

· 2,648 Views · 1 Like

Data Freshness: Definition, Alerts To Use, and Other Best Practices

Data freshness is how frequently data is updated and refreshed. Understand common failure points and grab SQL alerts you can integrate into your data pipelines.

May 29, 2023

· 2,957 Views · 1 Like

Testing, Monitoring, and Data Observability: What’s the Difference?

This article will give you a better understanding of data quality testing, monitoring, and observability. So let's explore these concepts together.

May 22, 2023

· 2,150 Views · 1 Like

Data Quality Survey: Resolving Data Issues Takes 166% Longer Year Over Year

The second annual State of Data Quality Survey revealed shocking statistics, such as how data quality impacts 31% of revenue on average. Find out what else we uncovered.

May 17, 2023

· 1,464 Views · 1 Like

7 Essential Data Quality Tests

Getting started with data quality testing? Here are the 7 must-have checks to improve data quality and ensure reliability for your most critical assets.

April 27, 2023

· 3,808 Views · 1 Like

Data Quality Faults With Your Data Vault

In this article, we’ll dive into data vault architecture, challenges and best practices for maintaining data quality; and how data observability can help.

March 30, 2023

· 3,047 Views · 1 Like

Silver Bullet or False Panacea? 3 Questions for Data Contracts

Three open questions data contracts still need to answer for engineering teams.

February 1, 2023

· 2,064 Views · 1 Like

The 31 Flavors of Data Lineage and Why Vanilla Doesn’t Cut It

This article goes over the four critical reasons why your data quality solution needs to have data lineage.

January 27, 2023

· 5,905 Views · 4 Likes

Why Data Cleaning Is Failing Your ML Models – And What To Do About It

When it comes to achieving model accuracy, data cleaning alone is insufficient. Here’s why.

December 19, 2022

· 4,771 Views · 1 Like

Data Quality Monitoring — You’re Doing It Wrong

Monitoring your “important” data only gets you so far. If you really want data quality coverage, it's time to go deep and broad with your data monitors.

October 3, 2022

· 4,595 Views · 3 Likes

How to Make Data Anomaly Resolution Less Cartoonish

Fixing broken data doesn’t have to be a game of whack-a-mole. Here’s how to speed up your data incident resolution

September 13, 2022

· 3,611 Views · 3 Likes

How Data Observability Helps Data Catalog Initiatives

Here’s why some of the best data teams are investing in data observability before kicking off a data catalog initiative.

September 7, 2022

· 5,910 Views · 1 Like

Using the Airflow ShortCircuitOperator to Stop Bad Data from Reaching ETL Pipelines

See how to leverage the Airflow ShortCircuitOperator to create data circuit breakers to prevent bad data from reaching your data pipelines.

September 3, 2022

· 6,878 Views · 1 Like

How to Build Spark Lineage for Data Lakes

Better understand the intricacies of Spark and how to build a lineage graph for data lakes.

August 8, 2022

· 4,367 Views · 1 Like

Why Building an External Data Product Is So Hard

Developing an external data product is different and, let's face it, more complex than serving internal customers.

Updated July 19, 2022

· 4,222 Views · 3 Likes

How to Build Reliability Into Your Data Stack

Data has become more integral to the day-to-day operations of modern business and powers digital products; the need for reliable data will only increase.

June 8, 2022

· 5,236 Views · 1 Like

Data Lakes, Warehouses and Lakehouses. Which is Best?

Everything you need to know about the foundation of your data infrastructure: data warehouses, data lakes, and the data bakehouse.

May 26, 2022

· 8,367 Views · 6 Likes

Comments

Data Lakes, Warehouses and Lakehouses. Which is Best?

Jul 22, 2022 · Lior Gavish

Redshift, Big Query and Snowflake are the cloud data warehouses I see with the most adoption, but Azure and Microsoft are of course huge players in this space as well. Thank you for the question.