Conda

Who

uses Conda?

Yeah, it's easy, right? So you get all your binaries there, you install all your other dependencies, and you're ready to go, right? But what if your Python dependency is not on conda, right? For us, for example, all our internal libraries, they're on an internal index, right? They're not on conda.

If I install them, it will mess up my dependencies tree, right? Like I don't know what I actually have in my, like my log file is not my log file anymore.

What Pixie actually is, it's a fusion of UV and conda.

It supports multiple platforms, multiple environments, multiple language.

So it's not just Python, it's actually like any language that you find on Conda.

And it uses Conda and UV to resolve dependencies.

We have a critical notice for all users.

The newest subminor image versions, such as two point one point one one seven and two point two point eight five, no longer contain pre-configured Conda channels.

Google is transitioning its default aliases to point to these new channel-less images.

If your Spark jobs rely on Conda to install third-party packages, you must manually configure your Conda channels during cluster initialization.

Furthermore, old clusters built on older subminor images will eventually need to be deleted and replaced to avoid breaking workloads.

On the bright side, Iceberg version one point one zero is now supported on Spark two point three images, and self join and skewed join optimizations are now enabled by default in new Compute Engine image versions.