data engineering - MEDevel.com: Open-source for Healthcare, and Education

data engineering

A collection of 80 posts

33 Free Data Visualization Libraries, Frameworks and Apps, Mastering Data Presentation

33 Free Data Visualization Libraries, Frameworks and Apps, Mastering Data Presentation

Welcome to an exhaustive list of over 30 data visualization libraries, frameworks, and applications. These tools span across a myriad of platforms and programming languages, providing you with the capability to present complex data in visually appealing and accessible ways. These solutions cater to a wide range of needs, whether

27 Free Open-source Text Annotation Solutions

27 Free Open-source Text Annotation Solutions

What is Text annotation? Text annotation is the process of associating labels or tags to specific parts of a text, such as phrases, words, or sentences. The aim is to provide additional information about the text, which can then be used for further analysis or processing, particularly in the field

Harvest News Like a Pro: Introducing News-Please, Your Open-Source Solution for News Extraction and Archiving

Harvest News Like a Pro: Introducing News-Please, Your Open-Source Solution for News Extraction and Archiving

news-please is an open-source news crawler that extracts structured information from news websites. It uses libraries like scrapy, Newspaper, and readability, and can follow internal hyperlinks and read RSS feeds to fetch both recent and archived articles. It also features a library mode for Python developers and can extract articles

Python Wizards: Extract Valuable Insights from Google Maps in Minutes with Google Maps Scraper

Python Wizards: Extract Valuable Insights from Google Maps in Minutes with Google Maps Scraper

Before we start, it is important to add the following Disclaimer by the project creators. Disclaimer for Google Maps Scraper Project This Google Maps Scraper is provided for educational and research purposes only. By using this Google Maps Scraper, you agree to comply with local and international laws regarding data

25 Best Free SQL Viewers and Clients for 2024

25 Best Free SQL Viewers and Clients for 2024

An SQL Viewer is a valuable tool for various professionals and students involved in working with databases, offering a convenient and efficient way to interact with and analyze data using SQL queries.

DataCleaner is a Free App Data Quality Analysis, Profiling, Cleansing, Duplicate Detection and more

DataCleaner is a Free App Data Quality Analysis, Profiling, Cleansing, Duplicate Detection and more

DataCleaner is a data quality analysis application and a solution platform for DQ solutions. It's core is a strong data profiling engine, which is extensible and thereby adds data cleansing, transformations, enrichment, deduplication, matching and merging. Features * Profiles and analyzes your database within minutes! * Access almost any datastore

Trowser: Free App to Browse Large Text Files with Ease

data engineering

Trowser: Free App to Browse Large Text Files with Ease

Trowser is a browser for large line-oriented text files, implemented in 3 alternate programming languages: Tcl/Tk, Python and C++/Qt. Compared to plain text viewers, trowser adds color highlighting, a persistent search history, graphical bookmarking and a separate search result window. The search window is especially designed to be

Talend Open Studio for Big Data: Streamlining ETL and ELT Processes

Talend Open Studio for Big Data: Streamlining ETL and ELT Processes

Talend Open Studio for Big Data is a powerful and versatile software solution designed to facilitate the integration and transformation of big data using Hadoop and NoSQL technologies. Whether you are working with massive datasets or complex data processing tasks, Talend Open Studio for Big Data provides the necessary tools

Understanding Big Data and How to Synthesize its Usage

Understanding Big Data and How to Synthesize its Usage

Think of the term Big Data as a way to funnel multiple data streams into one medium in order to analyze it. And by analysis, we mean fishing out trends as well as insights. This isn’t a new concept; it’s been around since the 1950s and has been

10 DataFrame Tricks with Pandas (Python Tutorial)

10 DataFrame Tricks with Pandas (Python Tutorial)

Pandas is an incredibly popular open-source data manipulation and analysis library for Python. It has gained immense popularity due to its ability to simplify complex data handling tasks. With Pandas, you can effortlessly work with various data structures and leverage a wide range of data analysis tools to manipulate and

CKAN: The Essential Open-Source Data Management Platform for Organizations and Governments

CKAN: The Essential Open-Source Data Management Platform for Organizations and Governments

CKAN is an open-source data management platform and self-hosted data portal that is widely used by various organizations and governments around the world. It plays a crucial role in facilitating the publication, management, and sharing of data. With CKAN, organizations and governments can effectively store, organize, and distribute their data,

19 Open-source Self-hosted Free Data Dashboard For Business Intelligence

19 Open-source Self-hosted Free Data Dashboard For Business Intelligence

What is a Data Dashboard A data dashboard for business intelligence is a powerful tool that enables organizations to make sense of their data and gain valuable insights. It provides a visual representation of key metrics, trends, and performance indicators, allowing users to monitor and analyze data in real-time. Benefits

14 Top Open-source Low-code and No-code Platforms for Building Internal Tools

14 Top Open-source Low-code and No-code Platforms for Building Internal Tools

What are Internal Tools? Internal tools refer to software applications or systems that are developed and used within an organization to support internal processes and operations. These tools are specifically designed to meet the unique needs and requirements of the organization. Why Enterprises Need Internal Tools? Enterprises may need to

14 Top Free and Open-source Log File Viewers

14 Top Free and Open-source Log File Viewers

What is a log file? A log file is a file that records events, actions, and system messages generated by various software applications, operating systems, or devices. It serves as a detailed record of activities and can be useful for troubleshooting, analysis, and auditing purposes. What is a log file

Qri is an Open-source Dataset Manager and Ogranizer

Qri is an Open-source Dataset Manager and Ogranizer

Qri CLI is a dataset version control system built on the distributed web

Seal Report is an Open-source Reporting Tool for .Net

Seal Report is an Open-source Reporting Tool for .Net

Seal Report is an open-source reporting tool for .NET that allows users to create and customize reports from various data sources. It offers a user-friendly interface, scheduling capabilities, and exporting options in different formats, making it ideal for data analysis and business intelligence. Features * Dynamic SQL sources: Use either your

Python: How to Convert JSON to CSV and CSV to JSON (Tutorial)

Python: How to Convert JSON to CSV and CSV to JSON (Tutorial)

There are several reasons why you might need to convert JSON to CSV and vice versa in Python: 1. Data Transformation: JSON and CSV are two common formats for storing and exchanging data. Converting between them allows you to transform and manipulate data in different ways based on your needs.

Read and write Text, JSON, and CSV files in Python

Read and write Text, JSON, and CSV files in Python

Python is widely recognized as a high-level programming language, lauded for its simplicity and readability. It enjoys extensive usage across diverse domains, including web development, data analysis, artificial intelligence, and scientific computing. Python's syntax empowers programmers to compose code that is clear and concise, making it an ideal

Data Visualization using Python Pandas library

Data Visualization using Python Pandas library

In this tutorial, we will explore how to use Pandas to visualize data. We will cover various techniques and code snippets to create insightful visualizations. Let's dive in! 1- Import the necessary libraries: import pandas as pd import matplotlib.pyplot as plt 2- Load the data into a

How to Filter Data with the Pandas Library (Python Tutorial)

How to Filter Data with the Pandas Library (Python Tutorial)

To filter data using Pandas, one effective approach is to utilize boolean indexing. This powerful technique allows you to select rows from a DataFrame based on specific conditions. By applying boolean indexing, you can easily extract the desired subset of data that meets certain criteria. Below, I have provided some

Tutorial: How to select Rows and Columns using Pandas (Python)

Tutorial: How to select Rows and Columns using Pandas (Python)

Pandas is a powerful open-source library for data manipulation and analysis in Python. It offers easy-to-use data structures and analysis tools, making it valuable for data scientists, analysts, and developers working with structured data. Install and start using Pandas Python Library for Data EngineeringPandas is a powerful and popular open-source

How to Convert JSON file to CSV in 2 Lines of Python Code

How to Convert JSON file to CSV in 2 Lines of Python Code

To convert JSON to CSV using Python, you can use the pandas library. What is Pandas library? Pandas is a powerful open-source library for data analysis and manipulation in Python. It offers data structures and functions that make it easy to efficiently manipulate and analyze structured data, including CSV files,

Orange is an Open-source Self-hosted Toolbox for Data Mining

Orange is an Open-source Self-hosted Toolbox for Data Mining

Orange is a powerful and user-friendly data mining and visualization toolbox designed for both beginners and experienced users. With Orange, you can easily explore and analyze your data without the need for any programming skills or advanced mathematical knowledge. The primary goal of Orange is to make data science accessible

RATH is an Open-source, Self-hosted Platform for Data Exploration, Visualization, and Analysis

RATH is an Open-source, Self-hosted Platform for Data Exploration, Visualization, and Analysis

RATH is not only an open-source alternative to data analysis and visualization tools like Tableau, but it goes beyond that. It revolutionizes the exploratory data analysis workflow by leveraging its augmented analytic engine to automatically uncover patterns, insights, and causal relationships. Moreover, it takes these discoveries a step further by

22 Open-source Database Visualization Panels and Dashboards for Business Intelligence (BI)

22 Open-source Database Visualization Panels and Dashboards for Business Intelligence (BI)

Database visualization panels are powerful tools that allow users to visually explore and analyze data stored in databases. These panels provide an intuitive interface to interact with database data and present it in a visually appealing and easy-to-understand manner. Features * Data exploration: Database visualization panels enable users to explore large

Kuwala: Open-source Self-hosted No-Code Data Workspace for Business Intelligence (BI) Analysts

Kuwala: Open-source Self-hosted No-Code Data Workspace for Business Intelligence (BI) Analysts

Kuwala is a data workspace that allows BI analysts and engineers to collaborate on building analytics workflows. It brings together data engineering tools like Airbyte, dbt, and Prefect into an intuitive interface. Kuwala emphasizes extendability, reproducibility, and enablement, empowering analysts and engineers to focus on their strengths. Key features include

13 Free and Open-source SQL Viewers and Editors

13 Free and Open-source SQL Viewers and Editors

SQL Viewer and Editor is a software tool that allows users to view and edit SQL queries. It provides a user-friendly interface for writing, executing, and managing SQL statements. With a SQL viewer and editor, users can interact with databases, run queries, and perform various tasks related to database management.

Chartbrew: Open-source Web App to Connect Databases and APIs

Chartbrew: Open-source Web App to Connect Databases and APIs

Open-source web platform used to create live reporting dashboards from APIs, MongoDB, Firestore, MySQL, PostgreSQL, and more 📈📊

14 Open-source Free Google Map Scrapping Tools and Scripts

14 Open-source Free Google Map Scrapping Tools and Scripts

Google Maps is a web mapping service developed by Google. It offers satellite imagery, street maps, panoramic views of streets, real-time traffic conditions, and route planning for traveling by foot, car, bicycle or public transportation. It is one of the most popular and widely used digital mapping services in the

Google Maps Scraper: Scrap Rich Data From Google Maps

Google Maps Scraper: Scrap Rich Data From Google Maps

This is a small lightweight Python + JavaScript project that enables you to scrap Google Map leads in almost no time. Features 1. Scrape up to 1200 Google Map Leads in just 25 minutes, providing you with an extensive pool of potential customers to drive sales. 2. Access 30 Data Points,

diskover: Open-source File Search and Indexer Tool For Data Engineers

diskover: Open-source File Search and Indexer Tool For Data Engineers

Diskover is an open source file system indexer that uses Elasticsearch to index and manage data across different storage systems. This means that Diskover is a powerful tool for system administrators to manage their storage infrastructure and make informed decisions about new infrastructure purchases. Diskover is a sustainable data management

DataSurgeon: Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Security Numbers and More

data engineering

DataSurgeon: Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Security Numbers and More

Quickly Extracts IP's, Email Addresses, Hashes, Files, Credit Cards, Social Secuirty Numbers and a lot More From Text

WebHarvest: Open-source Free Web Data Extraction Tool

data engineering

WebHarvest: Open-source Free Web Data Extraction Tool

Web data extraction (also known as web data mining or web scraping) is an incredibly useful tool for extracting valuable information from arbitrary web pages. It employs well-proven technologies such as XML and text processing to make the extraction process easy and efficient. With the help of web data extraction

Tad: Application for viewing and analyzing tabular data such as CSV files (Free app)

The Tad desktop application enables you to quickly view and explore tabular data in several of the most popular tabular data file formats: CSV, Parquet, and SQLite and DuckDb database files. Internally, the application is powered by an in-memory instance of DuckDb, a fast, embeddable database engine optimized for analytic