#Text-processing
Showing 49 of 49 repositories tagged #text-processing, ranked by stars
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
Diff Match Patch is a high-performance library in multiple languages that manipulates plain text.
Intuitive find & replace CLI (sed alternative)
fastNLP: A Modularized and Extensible NLP Framework. Currently still in incubation.
Python library for creating PEG parsers
Text Classification Algorithms: A Survey
A fast and convenient fuzzy matcher library for rust
The most accurate natural language detection library for Go, suitable for short text and mixed-language text
Program to convert lines of text into a tree structure.
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the Kreuzberg team. Kreuzberg is a fast, polyglot document intelligence engine with a Rust core. It extracts structured data from 56+ document formats using streaming parsers and built-in OCR.
A sharp cut(1) clone.
Natural language detection library for Go
Turn PDFs and EPUBs into audiobooks; subtitles or videos into dubbed videos (including translation), and more. For free. Pandrator uses local models, including voice-cloning (instant, RVC-enhanced, XTTS fine-tuning) and LLM processing. It aspires to be a user-friendly app with a GUI, an installer and all-in-one packages.
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).
Android accessibility service for AI-powered text transformation
π£οΈ Tool to generate adversarial text examples and test machine learning models against them
πΏ An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.
head/tail for structured data - summarize/preview JSON/YAML and source code
AI-writing humanizer for KO/EN/ZH/JA
Streamlining Text Processing
regex101 for the terminal β real-time matching, 3 engines, capture groups, replace mode, syntax highlighting, plain-English explanations, undo/redo, mouse support. Written in Rust.
This case study shows how to create a model for text analysis and classification and deploy it as a web service in Azure cloud in order to automatically classify support tickets. This project is a proof of concept made by Microsoft (Commercial Software Engineering team) in collaboration with Endava http://endava.com/en
A web app to create and browse text visualizations for automated customer listening.
Use Alfred to Control Superwhisper - AI Powered Voice to Text
Syllable-aware BPE tokenizer for the Amharic language (α ααα) β fast, accurate, trainable.
A small package to fuzzy match chinese words
efficient string matching in Golang via the aho-corasick algorithm.
Vision Framework IOS WWDC 2017
An LLM GUI application; enables you to interact with your files, offering dynamic parameters that can modify response behavior during runtime.
Transform and optimize your markdown documentation for Large Language Models (LLMs) and RAG systems. Generate llms.txt automatically.
AI-powered text compression library for RAG systems and API calls. Reduce token usage by up to 50-60% while preserving semantic meaning with advanced compression strategies.
Data science tooling for Racket
Repo for Applied Text Mining in Python (coursera) by University of Michigan
Translate books and big text files via Ollama LLMs using two-stage approach with initial translation, then AI self-reflection and refinement.
Python text processing, pattern matching, and NLP framework
Rust CLI implementing the Recursive Language Model (RLM) pattern for Claude Code. Process documents 100x larger than context windows through intelligent chunking, SQLite persistence, and recursive sub-LLM orchestration.
A Paste Stack & Paste Queue for Alfred. Pin & Reuse Text, Files, & More
Refiner improves your writing by correcting grammar and style, adjusting tone, and offering formatting options. It is useful for non-native speakers and professionals who communicate with text.
A text processing CLI tool that rewrites FS paths according to a pattern.
PowerShell βstandardβ library for supercharging your productivity. Provides a powerful cross-platform scripting environment enabling efficient analysis and sustainable science in myriad contexts.
A self-contained Rust reimplementation of uroman, a universal romanizer.
Python package providing an Inverted Index implementation using dictionaries
Can we quickly summarize & visualize text to get the details about the unstructured data? Yes we can! Please review this code pattern for all the steps involved to quickly summarize & visualize the data.
End to End Malicious URL Detection Project using Machine learning and deep learning
Sentiment Analysis using Convolution Neural Networks(CNN) and Google News Word2Vec
Applies diffs based on context, not line numbers. Useful for AI-generated code.
πΆ llm-distillery β’ use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.