data engineering - MEDevel.com

data engineering

A collection of 50 posts

14 Open-source Free Google Map Scrapping Tools and Scripts

14 Open-source Free Google Map Scrapping Tools and Scripts

Google Maps is a web mapping service developed by Google. It offers satellite imagery, street maps, panoramic views of streets, real-time traffic conditions, and route planning for traveling by foot, car, bicycle or public transportation. It is one of the most popular and widely used digital mapping services in the

Google Maps Scraper: Scrap Rich Data From Google Maps

Google Maps Scraper: Scrap Rich Data From Google Maps

This is a small lightweight Python + JavaScript project that enables you to scrap Google Map leads in almost no time. Features 1. Scrape up to 1200 Google Map Leads in just 25 minutes, providing you with an extensive pool of potential customers to drive sales. 2. Access 30 Data Points,

diskover: Open-source File Search and Indexer Tool For Data Engineers

diskover: Open-source File Search and Indexer Tool For Data Engineers

Diskover is an open source file system indexer that uses Elasticsearch to index and manage data across different storage systems. This means that Diskover is a powerful tool for system administrators to manage their storage infrastructure and make informed decisions about new infrastructure purchases. Diskover is a sustainable data management

DataSurgeon: Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Security Numbers and More

data engineering

DataSurgeon: Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Security Numbers and More

Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Secuirty Numbers and a lot More From Text

WebHarvest: Open-source Free Web Data Extraction Tool

data engineering

WebHarvest: Open-source Free Web Data Extraction Tool

Web data extraction (also known as web data mining or web scraping) is an incredibly useful tool for extracting valuable information from arbitrary web pages. It employs well-proven technologies such as XML and text processing to make the extraction process easy and efficient. With the help of web data extraction

Tad: Application for viewing and analyzing tabular data such as CSV files (Free app)

The Tad desktop application enables you to quickly view and explore tabular data in several of the most popular tabular data file formats: CSV, Parquet, and SQLite and DuckDb database files. Internally, the application is powered by an in-memory instance of DuckDb, a fast, embeddable database engine optimized for analytic

Bdash: Simple SQL Client for lightweight data analysis. (Free program)

Simple SQL Client for lightweight data analysis. Supported Databases * MySQL * PostgreSQL * Amazon Redshift * SQLite * Google BigQuery * Treasure Data * Amazon Athena Features * Run SQL queries * Save SQL queries * Draw charts * Share results Platforms macOS (Intel & Silicon) Windows, Lin

Mastering Data Manipulation with Python: A Comprehensive Guide to Reading and Writing Text, CSV, and JSON Files

Mastering Data Manipulation with Python: A Comprehensive Guide to Reading and Writing Text, CSV, and JSON Files

In the ever-expanding world of data-driven decision-making, Python is a powerful tool, providing developers and data enthusiasts with many libraries and functions for manipulating data files. One of the most fundamental tasks in data handling involves reading and writing files, and Python offers a versatile and efficient way to tackle

Open Metadata: Discover, Manage and Collaborate on your Data

data engineering

Open Metadata: Discover, Manage and Collaborate on your Data

OpenMetaData is a comprehensive platform that offers a range of functionalities, including data discovery, data lineage, data quality, observability, governance, and team collaboration. It is an open-source project that has gained immense popularity among companies across various industry verticals, thanks to its vibrant community and adoption. OpenMetaData is built on

Swiple: Open-source Automated Data Monitoring

Swiple: Open-source Automated Data Monitoring

The data quality automation plugin for data teams. Experience data quality observability in your ELT/ETL pipeline that would usually take a year to build, in just a few hours.

DQO: Open-source Data Quality Operations Center

DQO is a powerful DataOps friendly data quality monitoring tool that is designed to help you monitor and maintain the quality of your data. With DQO, you get access to a wide range of customizable data quality checks and data quality dashboards that make it easy to keep an eye

formsflow.ai: Low-code Workflow RAD Platform for Enterprise

formsflow.ai: Low-code Workflow RAD Platform for Enterprise

formsflow.ai is a low-code platform that helps build powerful business applications, quickly. It integrates open-source tools like form.io for forms, Camunda for workflows, Keycloak for security, and Redash for data analytics into a seamless and unified platform. Best of all, it's free!

Helical Insight CE: Open-source Business Intelligence (BI) Framework

Helical Insight CE: Open-source Business Intelligence (BI) Framework

Helical Insight is a revolutionary open-source business intelligence framework that can help you gain insights from one or multiple data sources, making it a versatile tool for your business needs. It offers a wide range of features that you would expect from any BI tool, be it open source or

22 Open-source Business Intelligence (BI) Dashboards

22 Open-source Business Intelligence (BI) Dashboards

Business Intelligence, commonly known as BI, is the process of collecting, analyzing, and presenting data to make informed business decisions. BI helps organizations to transform their raw data into meaningful insights that can drive their business strategies. BI provides a range of advantages to organizations, including improved decision-making, increased efficiency,

Lightdash: Is an Open-source Business Intelligence (BI) Dashboard

Lightdash: Is an Open-source Business Intelligence (BI) Dashboard

Lightdash instantly turns your dbt project into a full-stack BI platform. Analysts write metrics and Lightdash enables self-serve for the entire business.

Urungi: Open-source Business Intelligence and Reporting tool for SQL Database Engines

Urungi: Open-source Business Intelligence and Reporting tool for SQL Database Engines

Urungi is a powerful tool that is designed to help you easily create and manage statistics and dashboards, no matter where your data comes from. With Urungi, you can take advantage of a range of features that will help you get the most out of your data. To begin with,

CBoard Is an Open-source Self-hosted Business intelligence Reporting and Dashboard

CBoard Is an Open-source Self-hosted Business intelligence Reporting and Dashboard

CBoard is a highly versatile analysis platform that provides a plethora of features to users who are looking to improve their data analysis capabilities. It is an interactive platform that allows users to design multi-dimensional reports with ease. Not only is it beneficial for normal users, but it is also

Voici Generates Static Websites from Jupyter Notebooks

Voici Generates Static Websites from Jupyter Notebooks

Voici is an exceptional tool that offers a unique way of generating static dashboards from Jupyter Notebooks. Notably, it can replace Voilà with ease. This is because it provides and supports most of Voilà's configuration options and commands, while also offering some unique features of its own. One of the

Create Rich Report From Many Sources with pyreports

data engineering

Create Rich Report From Many Sources with pyreports

pyreports is a python library that allows you to create complex report from various sources

How to Scrap Webpages using Using Node.js and Cheerio

How to Scrap Webpages using Using Node.js and Cheerio

Web scraping is a process of extracting useful information from web pages. Node.js is a popular backend language that can be used for web scraping. In this tutorial, we will learn how to use Node.js for web scraping. Prerequisites Before we start, make sure you have the following

10 Top Open Source Privacy-First Web Analytics

10 Top Open Source Privacy-First Web Analytics

Google Analytics is a free web analytics service offered by Google that tracks and reports website traffic, allowing website owners to gain insights into the behavior of their users. It provides various tools and metrics to measure website performance, such as the number of visitors, their demographics, the pages they