A Python library to load structured table data from files/strings/URL with various data format: CSV / Excel / Google-Sheets / HTML / JSON / LDJSON / LTSV / Markdown / SQLite / TSV.
.. contents:: pytablereader :backlinks: top :depth: 2
Summary ========= pytablereader <https://github.com/thombashi/pytablereader>__ is a Python library to load structured table data from files/strings/URL with various data format: CSV / Excel / Google-Sheets / HTML / JSON / LDJSON / LTSV / Markdown / SQLite / TSV.
.. image:: https://badge.fury.io/py/pytablereader.svg :target: https://badge.fury.io/py/pytablereader :alt: PyPI package version
.. image:: https://img.shields.io/pypi/pyversions/pytablereader.svg :target: https://pypi.org/project/pytablereader :alt: Supported Python versions
.. image:: https://img.shields.io/pypi/implementation/pytablereader.svg :target: https://pypi.org/project/pytablereader :alt: Supported Python implementations
.. image:: https://github.com/thombashi/pytablereader/actions/workflows/lintandtest.yml/badge.svg :target: https://github.com/thombashi/pytablereader/actions/workflows/lintandtest.yml :alt: CI status of Linux/macOS/Windows
.. image:: https://coveralls.io/repos/github/thombashi/pytablereader/badge.svg?branch=master :target: https://coveralls.io/github/thombashi/pytablereader?branch=master :alt: Test coverage
.. image:: https://github.com/thombashi/pytablereader/actions/workflows/github-code-scanning/codeql/badge.svg :target: https://github.com/thombashi/pytablereader/actions/workflows/github-code-scanning/codeql :alt: CodeQL
Features
- Extract structured tabular data from various data format:
TM file - Google Sheets <https://www.google.com/intl/enus/sheets/about/> - HTML (`table tags) - JSON - Labeled Tab-separated Values (LTSV) __ - Line-delimited JSON(LDJSON) __ / NDJSON / JSON Lines - Markdown - MediaWiki - SQLite database file - Supported data sources are:
- Files on a local file system - Accessible URLs - str instances - Loaded table data can be used as:
- pandas.DataFrame __ instance - dict instance
Examples ========== Load a CSV table
:Sample Code: .. code-block:: python
import pytablereader as ptr import pytablewriter as ptw
# prepare data --- filepath = "sampledata.csv" csv_text = "\n".join([ '"attra","attrb","attr_c"', '1,4,"a"', '2,2.1,"bb"', '3,120.9,"ccc"', ])
with open(file_path, "w") as f: f.write(csv_text)
# load from a csv file --- loader = ptr.CsvTableFileLoader(file_path) for table_data in loader.load(): print("\n".join([ "load from file", "==============", "{:s}".format(ptw.dumpstabledata(tabledata)), ]))
# load from a csv text --- loader = ptr.CsvTableTextLoader(csv_text) for table_data in loader.load(): print("\n".join([ "load from text", "==============", "{:s}".format(ptw.dumpstabledata(tabledata)), ]))
:Output: .. code-block::
load from file ============== .. table:: sample_data
====== ====== ====== attra attrb attr_c ====== ====== ====== 1 4.0 a 2 2.1 bb 3 120.9 ccc ====== ====== ======
load from text ============== .. table:: csv2
====== ====== ====== attra attrb attr_c ====== ====== ====== 1 4.0 a 2 2.1 bb 3 120.9 ccc ====== ====== ======
Get loaded table data as pandas.DataFrame instance
:Sample Code: .. code-block:: python
import pytablereader as ptr
loader = ptr.CsvTableTextLoader( "\n".join([ "a,b", "1,2", "3.3,4.4", ])) for table_data in loader.load(): print(tabledata.asdataframe())
:Output: .. code-block::
a b 0 1 2 1 3.3 4.4
For more information
More examples are available at https://pytablereader.rtfd.io/en/latest/pages/examples/index.html
Installation ============
Install from PyPI
::
pip install pytablereader
Some of the formats require additional dependency packages, you can install the dependency packages as follows:
- Excel
- pip install pytablereader[excel]
- Google Sheets
- pip install pytablereader[gs]
- Markdown
- pip install pytablereader[md]
- Mediawiki
- pip install pytablereader[mediawiki]
- SQLite
- pip install pytablereader[sqlite]
- Load from URLs
- pip install pytablereader[url]
- All of the extra dependencies
- pip install pytablereader[all]
Install from PPA (for Ubuntu)
::
sudo add-apt-repository ppa:thombashi/ppa sudo apt update sudo apt install python3-pytablereader
Dependencies ============
- Python 3.7+
Python package dependencies (automatically installed) __
Optional Python packages
logging extras - loguru __: Used for logging if the package installed
excel extras - excelrd __
md extras - Markdown __
mediawiki extras - pypandoc __
sqlite extras - SimpleSQLite __
url extras - retryrequests __
pandas __ - required to get table data as a pandas data frame
lxml __ Optional packages (other than Python packages)
libxml2 (faster HTML conversion) pandoc __ (required when loading MediaWiki file) Documentation =============== https://pytablereader.rtfd.io/
Related Project =================
pytablewriter __ - Tabular data loaded by pytablereader can be written another tabular data format with pytablewriter.
Sponsors ==================================== .. image:: https://avatars.githubusercontent.com/u/44389260?s=48&u=6da7176e51ae2654bcfd22564772ef8a3bb22318&v=4 :target: https://github.com/chasbecker :alt: Charles Becker (chasbecker) .. image:: https://avatars.githubusercontent.com/u/46711571?s=48&u=57687c0e02d5d6e8eeaf9177f7b7af4c9f275eb5&v=4 :target: https://github.com/Arturi0 :alt: onetime: Arturi0 .. image:: https://avatars.githubusercontent.com/u/3658062?s=48&v=4 :target: https://github.com/b4tman :alt: onetime: Dmitry Belyaev (b4tman)
Become a sponsor