Amazon Redshift

Option B, Amazon DynamoDB.

Option C, Amazon Redshift.

And option D, Amazon Athena.

Let's pause here.

The correct answer here is option C, Amazon Redshift.

And here's why it's right.

Redshift is AWS's fully managed data warehouse, purpose-built for online analytical processing.

For a dedicated high performance warehouse handling heavy constant analytical queries with columnar storage, Redshift is the stronger fit here and it's one of the questions wordings that points to it.

The support of connectors.

There are Snowflake Databricks, BigQuery, Redshift, Dreamio, Spark, and generic ODBC LLDB connections.

So Mike, when I first started with Power BI, most of our things were done via ODBC.

And it was a pain in the gun because we had a virtual machine that allowed us to do so.

Most folks probably lie in the medium sized data, but Pandas definitely tops out.

I mean, he does some interesting benchmarks in here, gives a couple of good code examples, actually shows a really interesting post from Amazon Redshift team.

where they were looking at the composition of many of the tables that are out there in the Redshift environment.

If anybody's going to have a good view on what the size of data is and what big data could be, they're probably the ones to look at that.

Corre sobre una arquitectura de mezcla de expertos con trescientos veinte mil millones de parámetros y licencia MIT, devolviendo un rendimiento de codificación de nivel frontera a manos abiertas a una fracción del costo de los modelos cerrados.

También en el plano de infraestructura, AWS y Stardog lanzaron esta semana una capa semántica para IA agéntica, que permite a los agentes consultar bases de datos de Aurora y Redshift directamente sin ningún pipeline ETL.

La integración se apoya en Bedrock Agent Core y preserva el contexto relacional, eliminando un cuello de botella importante en los despliegues de agentes empresariales.

Daniel, ¿qué hay de nuevo en los frentes de datos y diseño?

[whooshing] The last one from OpenAI is their new data agent for ChatGPT Work.

The agent is designed for handling proprietary data within an organization with connections to data providers like Amazon Redshift, Datadog, Google BigQuery, ClickHouse, Databricks, MongoDB, and Snowflake.

OpenAI says the features can be used to ingest sales data and generate insights around core metrics like sales conversions and retention.

The goal is to realize the promise of being able to talk to your data and perform real analysis without needing to touch additional tools.

... uh, presumably on these, these kind of things, where you can query your S3, uh, data directly through DuckDB.

Um, of course there is, you know, we cannot let this go without mentioning Redshift, and I think we've all known for a long time that, you know, Redshift is definitely not the product that you recommend to, um, anyone [laughs] if you're not already a Redshift user or like you have to be on AWS for your, your columnar database.

I think even, uh, maybe you mentioned this, Mady, uh, as a kind of a, uh, a joke or funny because it's true that, uh, AWS people will recommend Snowflake on AWS, um, because it's just a better-

Yeah

There is many ways.

Uh, data pipeline, as you mentioned, there is even more, and so there is some competition, uh, happening between the teams.

And yes, uh, in the past, you know, there is al- also a lot of saying that, um, success of Snowflake is due to Redshift's failure because AWS was making a lot of good margin on the compute selling, uh, Snowflake rather than, than Redshift.

Yeah.

It is what it is.

Yeah, we had, we had queries that would run like 75 minutes and, and, and against Redshift, which was supposedly good.

But, you know, we, we didn't-

Yeah

... again, it's like, that's the other thing is that when, when you do the, the physical modeling, and that's why it's actually these layers, like conceptual, logical, and physical are quite important, is that y- when, well, when you're going from logic- logical to physical, you kind of need to think about, am I building a dimensional model or data vault or whatever methodology I want to use?

Yeah.

But also, what is my environment looking like? Am I doing Snowflake, Databricks, or Redshift or BigQuery or...

Because, like, you need to, you need to take some, um, optimization decisions at that level.

Stitch and Fivetran were really the two first modern data stack native data ingestion tools.

They were originally like load data from Salesforce into Redshift.

So I'd known George and Taylor forever.

And he collaborated often over the years.

I think it was 2018.

It was the first real cloud-native data warehouse that I used.

I used Redshift before, but I would call that kind of a different thing.

And the fact that Snowflake could do what it could do was magic it was just uh and and every single time you know their sequel coverage was actually not that good back then and uh so every time that they would release a new window function i would like read the release notes with bated breath i was like oh my god they now support rolling averages yet now we are whatever eight years beyond that and The idea that you would have a cloud-native SQL engine that reads from blob storage and has scale-out compute and has complete coverage of the SQL dialect, all of that is kind of expected.