Topics

#css #go #html #javascript #js #json #minifier #minify #svg #xml

Language breakdown

Go 94.4%

TypeScript 2.8%

JavaScript 1.1%

Python 0.8%

Shell 0.6%

Makefile 0.3%

▸ Files click to expand

README

Minify

Online demo if you need to minify files now.

Binaries of CLI for various platforms. See CLI for more installation instructions.

Windows binary from scoop install with scoop install main/minify

Python bindings install with pip install tdewolff-minify

JavaScript bindings install with npm i @tdewolff/minify

.NET bindings install with Install-Package NMinify or dotnet add package NMinify, thanks to Jonas Kamsker for the port

Did you know that the shortest valid piece of HTML5 is <!doctype html><title>x</title>? See for yourself at the W3C Validator!

Minify is a minifier package written in [Go][1]. It provides HTML5, CSS3, JS, JSON, SVG and XML minifiers and an interface to implement any other minifier. Minification is the process of removing bytes from a file (such as whitespace) without changing its output and therefore shrinking its size and speeding up transmission over the internet and possibly parsing. The implemented minifiers are designed for high performance (see https://github.com/privatenumber/minification-benchmarks where this library is (one of) the fastest JS minifiers).

The core functionality associates mimetypes with minification functions, allowing embedded resources (like CSS or JS within HTML files) to be minified as well. Users can add new implementations that are triggered based on a mimetype (or pattern), or redirect to an external command (like ClosureCompiler, UglifyCSS, ...).

Roadmap

[ ] Use ASM/SSE to further speed-up core parts of the parsers/minifiers
[x] Improve JS minifiers by shortening variables and proper semicolon omission
[ ] Speed-up SVG minifier, it is very slow
[x] Proper parser error reporting and line number + column information
[ ] Generation of source maps (uncertain, might slow down parsers too much if it cannot run separately nicely)
[ ] Create a cmd to pack webfiles (much like webpack), ie. merging CSS and JS files, inlining small external files, minification and gzipping. This would work on HTML files.

Prologue

Minifiers or bindings to minifiers exist in almost all programming languages. Some implementations are merely using several regular expressions to trim whitespace and comments (even though regex for parsing HTML/XML is ill-advised, for a good read see Regular Expressions: Now You Have Two Problems). Some implementations are much more profound, such as the YUI Compressor and Google Closure Compiler for JS. As most existing implementations either use JavaScript, use regexes, and don't focus on performance, they are pretty slow.

This minifier proves to be that fast and extensive minifier that can handle HTML and any other filetype it may contain (CSS, JS, ...). It is usually orders of magnitude faster than existing minifiers.

Installation

Make sure you have Git and Go (1.18 or higher) installed, run

mkdir Project
cd Project
go mod init
go get -u github.com/tdewolff/minify/v2

Then add the following imports to be able to use the various minifiers

go import ( 	"github.com/tdewolff/minify/v2" 	"github.com/tdewolff/minify/v2/css" 	"github.com/tdewolff/minify/v2/html" 	"github.com/tdewolff/minify/v2/js" 	"github.com/tdewolff/minify/v2/json" 	"github.com/tdewolff/minify/v2/svg" 	"github.com/tdewolff/minify/v2/xml" )

You can optionally run go mod tidy to clean up the go.mod and go.sum files.

See CLI tool for installation instructions of the binary.

Docker

If you want to use Docker, please see https://hub.docker.com/r/tdewolff/minify.

$ docker run -it tdewolff/minify --help

API stability

There is no guarantee for absolute stability, but I take issues and bugs seriously and don't take API changes lightly. The library will be maintained in a compatible way unless vital bugs prevent me from doing so. There has been one API change after v1 which added options support and I took the opportunity to push through some more API clean up as well. There are no plans whatsoever for future API changes.

Testing

For all subpackages and the imported parse package, test coverage of 100% is pursued. Besides full coverage, the minifiers are fuzz tested using github.com/dvyukov/go-fuzz, see the wiki for the most important bugs found by fuzz testing. These tests ensure that everything works as intended and that the code does not crash (whatever the input). If you still encounter a bug, please file a bug report!

Performance

The benchmarks directory contains a number of standardized samples used to compare performance between changes. To give an indication of the speed of this library, I've ran the tests on my Thinkpad T460 (i5-6300U quad-core 2.4GHz running Arch Linux) using Go 1.15.

name time/op CSS/sample_bootstrap.css-4 2.70ms ± 0% CSS/sample_gumby.css-4 3.57ms ± 0% CSS/sample_fontawesome.css-4 767µs ± 0% CSS/sample_normalize.css-4 85.5µs ± 0% HTML/sample_amazon.html-4 15.2ms ± 0% HTML/sample_bbc.html-4 3.90ms ± 0% HTML/sample_blogpost.html-4 420µs ± 0% HTML/sample_es6.html-4 15.6ms ± 0% HTML/sample_stackoverflow.html-4 3.73ms ± 0% HTML/sample_wikipedia.html-4 6.60ms ± 0% JS/sample_ace.js-4 28.7ms ± 0% JS/sample_dot.js-4 357µs ± 0% JS/sample_jquery.js-4 10.0ms ± 0% JS/sample_jqueryui.js-4 20.4ms ± 0% JS/sample_moment.js-4 3.47ms ± 0% JSON/sample_large.json-4 3.25ms ± 0% JSON/sample_testsuite.json-4 1.74ms ± 0% JSON/sample_twitter.json-4 24.2µs ± 0% SVG/sample_arctic.svg-4 34.7ms ± 0% SVG/sample_gopher.svg-4 307µs ± 0% SVG/sample_usa.svg-4 57.4ms ± 0% SVG/sample_car.svg-4 18.0ms ± 0% SVG/sample_tiger.svg-4 5.61ms ± 0% XML/sample_books.xml-4 54.7µs ± 0% XML/sample_catalog.xml-4 33.0µs ± 0% XML/sample_omg.xml-4 7.17ms ± 0%

name speed CSS/sample_bootstrap.css-4 50.7MB/s ± 0% CSS/sample_gumby.css-4 52.1MB/s ± 0% CSS/sample_fontawesome.css-4 61.2MB/s ± 0% CSS/sample_normalize.css-4 70.8MB/s ± 0% HTML/sample_amazon.html-4 31.1MB/s ± 0% HTML/sample_bbc.html-4 29.5MB/s ± 0% HTML/sample_blogpost.html-4 49.8MB/s ± 0% HTML/sample_es6.html-4 65.6MB/s ± 0% HTML/sample_stackoverflow.html-4 55.0MB/s ± 0% HTML/sample_wikipedia.html-4 67.5MB/s ± 0% JS/sample_ace.js-4 22.4MB/s ± 0% JS/sample_dot.js-4 14.5MB/s ± 0% JS/sample_jquery.js-4 24.8MB/s ± 0% JS/sample_jqueryui.js-4 23.0MB/s ± 0% JS/sample_moment.js-4 28.6MB/s ± 0% JSON/sample_large.json-4 234MB/s ± 0% JSON/sample_testsuite.json-4 394MB/s ± 0% JSON/sample_twitter.json-4 63.0MB/s ± 0% SVG/sample_arctic.svg-4 42.4MB/s ± 0% SVG/sample_gopher.svg-4 19.0MB/s ± 0% SVG/sample_usa.svg-4 17.8MB/s ± 0% SVG/sample_car.svg-4 29.3MB/s ± 0% SVG/sample_tiger.svg-4 12.2MB/s ± 0% XML/sample_books.xml-4 81.0MB/s ± 0% XML/sample_catalog.xml-4 58.6MB/s ± 0% XML/sample_omg.xml-4 159MB/s ± 0%

HTML

HTML (with JS and CSS) minification typically shaves off about 10%.

The HTML5 minifier uses these minifications:

strip unnecessary whitespace and otherwise collapse it to one space (or newline if it originally contained a newline)
strip superfluous quotes, or uses single/double quotes whichever requires fewer escapes
strip default attribute values and attribute boolean values
strip some empty attributes
strip unrequired tags (html, head, body, ...)
strip unrequired end tags (tr, td, li, ... and often p)
strip default protocols (http:, https: and `)


strip all comments (including conditional comments, old IE versions are not supported anymore by Microsoft)

shorten doctype and meta charset


lowercase tags, attributes and some values to enhance gzip compression


Options:

KeepSpecialComments preserve all special comments, including Server Side Includes such as and IE conditional comments such as and , see https://msdn.microsoft.com/en-us/library/ms537512(v=vs.85).aspx#syntax

KeepDefaultAttrVals preserve default attribute values such as // Faulty JS

	req := httptest.NewRequest(http.MethodGet, "/", nil)
	rec := httptest.NewRecorder()
	m.Middleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/html")
		,  = w.Write([]byte(input))
if err = w.(io.Closer).Close(); err != nil { 			panic(err) 		} 	})).ServeHTTP(rec, req) } <pre><code class="lang-">#### ResponseWriter</code></pre> go func Serve(w http.ResponseWriter, r *http.Request) { 	mw := m.ResponseWriter(w, r) 	defer mw.Close() 	w = mw
http.ServeFile(w, r, path.Join("www", r.URL.Path)) } <pre><code class="lang-">#### Custom response writer ResponseWriter example which returns a ResponseWriter that minifies the content and then writes to the original ResponseWriter. Any write after applying this filter will be minified.</code></pre> go type MinifyResponseWriter struct { 	http.ResponseWriter 	io.WriteCloser }
func (m MinifyResponseWriter) Write(b []byte) (int, error) { 	return m.WriteCloser.Write(b) }
// MinifyResponseWriter must be closed explicitly by calling site. func MinifyFilter(mediatype string, res http.ResponseWriter) MinifyResponseWriter { 	m := minify.New() 	// add minfiers
mw := m.Writer(mediatype, res) 	return MinifyResponseWriter{res, mw} } <pre><code class="lang-"></code></pre> go // Usage func(w http.ResponseWriter, req *http.Request) { 	w = MinifyFilter("text/html", w) 	if _, err := io.WriteString(w, "<p class="message"> This HTTP response will be minified. </p>"); err != nil { 		panic(err) 	} 	if err := w.Close(); err != nil { 		panic(err) 	} 	// Output: <p class=message>This HTTP response will be minified. } <pre><code class="lang-">### Templates

Here's an example of a replacement for template.ParseFiles from template/html, which automatically minifies each template before parsing it.


Be aware that minifying templates will work in most cases but not all. Because the HTML minifier only works for valid HTML5, your template must be valid HTML5 of itself. Template tags are parsed as regular text by the minifier.</code></pre> go func compileTemplates(filenames ...string) (*template.Template, error) { 	m := minify.New() 	m.AddFunc("text/html", html.Minify)
var tmpl *template.Template 	for _, filename := range filenames { 		name := filepath.Base(filename) 		if tmpl == nil { 			tmpl = template.New(name) 		} else { 			tmpl = tmpl.New(name) 		}
b, err := os.ReadFile(filename) 		if err != nil { 			return nil, err 		}

mb, err := m.Bytes("text/html", b) if err != nil { return nil, err } tmpl.Parse(string(mb)) } return tmpl, nil } <pre><code class="lang-">Example usage:</code></pre> go templates := template.Must(compileTemplates("view.html", "home.html")) `


FAQ
Newlines remain in minified output
While you might expect the minified output to be on a single line for it to be fully minified, this is not true. In many cases, using a literal newline doesn't affect the file size, and in some cases it may even reduce the file size.
A typical example is HTML. Whitespace is significant in HTML, meaning that spaces and newlines between or around tags may affect how they are displayed. There is no distinction between a space or a newline and they may be interchanged without affecting the displayed HTML. Remember that a space (0x20) and a newline (0x0A) are both one byte long, so that there is no difference in file size when interchanging them. This minifier removes unnecessary whitespace by replacing stretches of spaces and newlines by a single whitespace character. Specifically, if the stretch of white space characters contains a newline, it will replace it by a newline and otherwise by a space. This doesn't affect the file size, but may help somewhat for debugging or file transmission objectives.

Another example is JavaScript. Single or double quoted string literals may not contain newline characters but instead need to escape them as \n`. These are two bytes instead of a single newline byte. Using template literals it is allowed to have literal newline characters and we can use that fact to shave-off one byte! The result is that the minified output contains newlines instead of escaped newline characters, which makes the final file size smaller. Of course, changing from single or double quotes to template literals depends on other factors as well, and this minifier makes a calculation whether the template literal results in a shorter file size or not before converting a string literal.