One List to Rule Them All

A single-process Go service that combines an unordered set of public uBlock Origin filter-list URLs into one stable subscription URL. It uses bbolt and atomically installed local files; it has no accounts, JavaScript, external database, queue, cache, or scheduler.

Build and run

Go 1.24 or newer is required.

go build -trimpath -o one-list ./cmd/one-list
./one-list serve --listen-address :8080 --public-base-url https://lists.example.com --data-dir ./data

Open http://localhost:8080, paste one source URL per line, and add the resulting /lists/{id}.txt URL to uBlock Origin as a custom list.

Command line

The executable supports these commands:

one-list serve [flags]  Start the HTTP service.
one-list version        Print the build version.
one-list --version      Print the build version.

Use one-list serve -h, one-list serve -help, or one-list serve --help to print the generated flag reference. Both -flag value and --flag value forms are accepted; --flag=value is also accepted. Boolean flags can be enabled without a value, for example --allow-http-sources, or set explicitly with --allow-http-sources=false.

Runtime configuration

Every user-configurable runtime setting has a command-line flag and an environment-variable equivalent. An explicitly supplied flag takes precedence over its environment variable, and the documented default is used when neither is supplied.

Command-line flag	Environment variable	Default	Description
--listen-address	LISTEN_ADDRESS	:8080	HTTP server listen address. Must not be empty.
--public-base-url	PUBLIC_BASE_URL	empty	Absolute public HTTP(S) base URL used in generated subscription links. It must not contain credentials, a query, or a fragment. When empty, the service derives the base URL from each request. A trailing slash is removed.
--data-dir	DATA_DIR	./data	Persistent data directory. Must not be empty.
--allow-http-sources	ALLOW_HTTP_SOURCES	false	Permit plain HTTP source URLs. Enabling this also adds port 80 to the allowed source ports.
--allowed-source-ports	ALLOWED_SOURCE_PORTS	443	Comma-separated source ports. Each port must be an integer from 1 through 65535, and the resulting set must not be empty.
--max-form-body-bytes	MAX_FORM_BODY_BYTES	65536 (64 KiB)	Maximum URL-encoded form request body size, in bytes.
--max-source-urls	MAX_SOURCE_URLS	20	Maximum number of source URLs in one combined list.
--max-compressed-source-bytes	MAX_COMPRESSED_SOURCE_BYTES	20971520 (20 MiB)	Maximum compressed response size for one source, in bytes.
--max-decompressed-source-bytes	MAX_DECOMPRESSED_SOURCE_BYTES	104857600 (100 MiB)	Maximum decompressed response size for one source, in bytes.
--max-output-bytes	MAX_OUTPUT_BYTES	262144000 (250 MiB)	Maximum generated combined-list size, in bytes.
--max-line-bytes	MAX_LINE_BYTES	4194304 (4 MiB)	Maximum source line size, in bytes.
--max-include-depth	MAX_INCLUDE_DEPTH	4	Maximum nested !#include depth. A direct include has depth 1.
--dns-timeout	DNS_TIMEOUT	5s	DNS resolution timeout.
--connect-timeout	CONNECT_TIMEOUT	5s	TCP connection timeout.
--tls-handshake-timeout	TLS_HANDSHAKE_TIMEOUT	5s	TLS handshake timeout.
--response-header-timeout	RESPONSE_HEADER_TIMEOUT	10s	Response-header timeout.
--source-request-timeout	SOURCE_REQUEST_TIMEOUT	30s	Timeout for one complete source request.
--generation-timeout	GENERATION_TIMEOUT	1m	Timeout for one complete generation or refresh.
--max-redirects	MAX_REDIRECTS	5	Maximum redirects followed for one source request. May be zero.
--refresh-interval	REFRESH_INTERVAL	24h	Age after which a generated list is refreshed when requested.
--refresh-retry-cooldown	REFRESH_RETRY_COOLDOWN	15m	Minimum delay before retrying a failed refresh.
--max-concurrent-generations	MAX_CONCURRENT_GENERATIONS	4	Maximum simultaneous list generations or refreshes.
--global-fetch-concurrency	GLOBAL_FETCH_CONCURRENCY	20	Maximum simultaneous source fetches across the service.
--per-host-fetch-concurrency	PER_HOST_FETCH_CONCURRENCY	2	Maximum simultaneous source fetches to one host.
--shutdown-timeout	SHUTDOWN_TIMEOUT	15s	Graceful HTTP shutdown timeout after SIGINT or SIGTERM.
--log-level	LOG_LEVEL	info	Log level: debug, info, warn, or error.

Byte limits are plain base-10 integers; suffixes such as MiB are not accepted as values. Durations use Go duration syntax, such as 500ms, 30s, 15m, 2h, or combinations such as 1h30m; there is no d unit, so use 24h for one day. Duration and limit values must be positive, except --max-redirects, which may be zero. Boolean environment variables accept the forms supported by Go's strconv.ParseBool: 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, and False.

For example, the environment can provide defaults while a flag overrides one setting:

LISTEN_ADDRESS=:9000 \
PUBLIC_BASE_URL=https://lists.example.com \
DATA_DIR=/var/lib/one-list \
./one-list serve --log-level debug

HTTPS sources on port 443 are the default. Plain HTTP must be explicitly enabled with --allow-http-sources or ALLOW_HTTP_SOURCES=true; port 80 is then added to the allowed set. Do not enable arbitrary source ports unless they are required.

Build configuration

The Makefile accepts these build-time variables either from the environment or as make arguments. They do not configure a running service.

Make variable	Default	Description
VERSION	dev	Version embedded in the binary and printed by one-list version or one-list --version.
ARTIFACT_SUFFIX	empty	Suffix added to filenames produced by make cross-build.
BIN_DIR	bin	Directory in which build artifacts are written.

Persistence, backup, and restore

The data directory is the complete backup unit:

data/
├── one-list.db
├── lists/
└── tmp/

Run one application instance per data directory. For a consistent offline backup, stop the process and copy the entire directory. Restore that directory before restarting. Generated files are installed by same-filesystem rename and never edited in place. Abandoned service temporary files older than 24 hours are removed at startup.

Refresh behavior

Lists refresh synchronously on subscription requests after the configured refresh interval (24 hours by default). There is no scheduler or detached work. One per-ID lock prevents duplicate work, a global semaphore bounds generations, and failed refreshes retain and serve the last good file. The refresh-attempt timestamp provides a retry cooldown (15 minutes by default). Conditional If-None-Match evaluation occurs only after any required refresh.

Security boundaries

Source URLs are hostile input. The fetcher only permits configured schemes and ports, rejects credentials, resolves every host itself, rejects every result if any resolved IP is loopback/private/link-local/multicast/unspecified/carrier-grade-NAT/reserved/documentation space, and dials only a validated numeric address. TLS still uses the original hostname for SNI and certificate verification. Redirect destinations receive the same validation. The service does not proxy cookies, authorization, forwarding, or client headers.

The form body, source count, compressed body, decompressed body, line length, include depth, generated output, redirects, fetch concurrency, and all network phases are bounded. Only UTF-8 identity or gzip source bodies are accepted. Aggregation IDs are exact lowercase SHA-256 hex strings before they reach filesystem paths. HTML is rendered with html/template, source contents are never logged, and no shell is executed.

DNS validation and validated dialing prevent the usual resolve-then-rebind gap inside one request. This is still an outbound public-web fetcher: deploy it with OS-level egress policy as defense in depth, and do not enable arbitrary ports unless required.

Determinism and uBlock compatibility

Canonical URLs are deduplicated and bytewise sorted before their JSON SHA-256 ID is calculated. Compilation always follows that canonical order, normalizes newlines, strips source metadata/comments at top level, preserves the first exact occurrence of ordinary top-level rules, and emits a deterministic header with exactly one final newline.

!#if, !#else, and !#endif blocks are preserved without deduplicating their contents. Relative !#include directives are recursively fetched and replaced in place before compilation, including when they occur inside conditional blocks. Include paths resolve from the fetched list's final URL; absolute URLs, absolute paths, and parent-directory traversal are rejected, and cycles or nesting beyond --max-include-depth fail generation. The service does not semantically normalize rules or implement the full uBlock parser.

Service manager example

[Unit]
Description=One List to Rule Them All
After=network-online.target

[Service]
User=one-list
Group=one-list
ExecStart=/usr/local/bin/one-list serve --listen-address :8080 --public-base-url https://lists.example.com --data-dir /var/lib/one-list
Restart=on-failure
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/var/lib/one-list

[Install]
WantedBy=multi-user.target

Terminate with SIGINT or SIGTERM for bounded graceful shutdown. Build release binaries with make cross-build VERSION=1.0.0.

Releases

Every push to dev runs the test suite in Woodpecker. Pushing a tag in the form v{major}.{minor}.{patch} runs the same test suite before the release build. The release pipeline then cross-compiles every target from the Makefile and attaches the versioned binaries and a SHA-256 checksum file to the matching forge release. For example, v1.2.3 produces one-list-linux-amd64-v1.2.3 and the equivalent artifacts for the other targets.

Configure a Woodpecker repository secret named forge_token with a Forgejo/Gitea API token that can write releases for this repository.

License

Copyright (C) 2026 Maroš Kučera.

One List to Rule Them All is free software licensed under the GNU General Public License, version 3 only. The complete license text is included in COPYING.

Authors

• Maroš Kučera