New post about how to write data from a Apache Spark DataFrame into a Elasticsearch/Opensearch database #datascience #databricks #elasticsearch #opensearch #bigdata #apachespark #spark #tech #programming #python:

https://pedro-faria.netlify.app/posts/2025/2025-03-16-spark-elasticsearch/en/

Easier to use: DuckDB gets local web user interface

As of version 1.2.1, the DuckDB in-process database can be conveniently operated via a local UI, which is installed as an extension, as an alternative to CLI.

https://www.heise.de/en/news/Easier-to-use-DuckDB-gets-local-web-user-interface-10316323.html?wt_mc=sm.red.ho.mastodon.mastodon.md_beitraege.md_beitraege&utm_source=mastodon

Einfacher bedienen: DuckDB erhält lokale Web-Benutzeroberfläche

Die In-Process-Datenbank DuckDB lässt sich ab Version 1.2.1 alternativ zur CLI komfortabel über ein lokales UI bedienen, das als Extension installiert wird.

https://www.heise.de/news/Einfacher-bedienen-DuckDB-erhaelt-lokale-Web-Benutzeroberflaeche-10316264.html?wt_mc=sm.red.ho.mastodon.mastodon.md_beitraege.md_beitraege&utm_source=mastodon

Unleash the power of Big Data with https://xcware.com! Our Sky Computing platform accelerates Apache Spark workloads, optimizing performance across multi-cloud environments. Process massive datasets faster, smarter, and without vendor lock-in. #BigData #ApacheSpark #SkyComputing #CloudComputing #AI #DevOps #MLOps #News #EdgeAI

In the world of data science, raw data serves as the foundation for generating actionable insights. However, managing, processing, and transforming this data into a usable format requires specialized tools.

#ApacheSpark #ApacheHadoop #Kafka #Airflow #DBT #Presto #SQL #Python #PySpark #Snowflake #Redshift #BigQuery #Kubernetes #Docker #Terraform #Databricks

read more: https://reshukhushi.wordpress.com/2024/12/03/top-data-engineering-tools-for-data-science-professionals/

Spark Connect is revolutionizing the way we run Spark applications. With version 3.4 and beyond, remote client applications written in Scala or Python can now run on a Spark cluster, offering more flexibility than ever before. Read Sergey Kotlov's latest article now.

#ApacheSpark #DataEngineering

https://towardsdatascience.com/adopting-spark-connect-cdd6de69fa98

Apache Spark 3 - Spark Programming in Python for Beginners

Data Engineering using PySpark

https://couponfrogg.com/coupons/apache-spark-programming-in-python-for-beginners/

Hey Mastodon! I created a "content directory" for all my content across Linkedin, Twitter, and YouTube.

I spent about 60 hours engineering this over the last 2 weeks.

You can see my content split into categories like #dataengineering #apachespark, #apacheairflow, etc.

I'll be adding a search function soon too so you can find exactly what you're looking for from the mountain of content I've created over the last two years!

Check it out here!

https://www.eczachly.com/about