How we test providers

Last updated: 2026-10-15 Independently verified ~9 min read

progeofde Editorial

Editor & methodology owner — 6 years in hosting, 3 in GPU cloud. Hosts an active homelab, pays for every test account out-of-pocket.

1. The principles

Reproduction over rhetoric. Anyone reading the page should be able to run our commands and get within 10% of our numbers.
No vendor money touches the verdict. Affiliate links are present, but they do not change which provider we recommend. See disclosure.
Date-stamp everything. "Observed on YYYY-MM-DD" is the only price we publish. Renewal prices, IPv4 add-ons, and bandwidth policies change without notice.
Distinguish measured vs claimed. "Provider claims 5 Gbps port" is different from "we measured 4.2 Gbps single-stream TCP throughput from Frankfurt to Mumbai for 30 minutes."
Show the boring data. Raw CSVs, screenshots, command output. This is the boring part that makes the rest of the site trustable.

2. VPS benchmarking (CPU, RAM, disk, network)

Test box standard

Every VPS is provisioned in the provider's cheapest plan with at least 1 vCPU, 1 GB RAM, 25 GB SSD. We use the smallest viable box because that's the segment most readers are buying. Where a provider charges for IPv4 separately, we add it; we do not test IPv6-only because the audience needs both.

CPU and RAM

# Single-threaded baseline
sysbench cpu --cpu-max-prime=20000 --time=60 run

# Memory bandwidth
sysbench memory --memory-block-size=1M --memory-total-size=10G run

# 7zip single-threaded composite (MIPS)
7zr b -mmt1

We run each benchmark three times after a 5-minute warm-up, discard the worst result, and report the median.

Disk

# Sequential read/write at 4K and 1M block sizes
fio --name=rand --ioengine=libaio --direct=1 \
    --filename=/tmp/fiotest --bs=4k --size=2G \
    --rw=randrw --runtime=60 --numjobs=4 --iodepth=32

# 4K random IOPS under saturation (the real-world proxy)
fio --name=iops --ioengine=libaio --direct=1 \
    --filename=/tmp/fioiops --bs=4k --size=4G \
    --rw=randread --runtime=60 --numjobs=4 --iodepth=64

Network

Single-stream TCP throughput using iperf3 against our probe in Frankfurt (IPv4 + IPv6, 60 seconds, 3 trials). We also test the outbound bandwidth quota by downloading from a 1 GB object for as long as the provider allows before throttling or charging overage.

3. Latency and routing

For regional articles (Japan, Singapore, Hong Kong, China) we run ping and tcping from three vantage points:

A RIPE Atlas probe in the country (if available) or a paid probe in a regional data center.
A home consumer connection (Verizon FIOS, US East) to test "from the audience."
A Linode/Akamai nanode in Tokyo / Singapore / Frankfurt to test the cross-provider routing.

For China routing we use nexttrace to record the AS path; we report which backbone hops the traffic crosses (CN2 / CMIN2 / 9929 / standard 163) when this is verifiable from the public traceroute.

4. GPU benchmarks

For GPU cloud articles we use a workload suite that mirrors what people actually rent GPUs for:

Stable Diffusion XL — 100 images at 1024×1024, 30 steps, FP16, measure seconds-per-image.
LLM inference (vLLM) — Llama 3.1 8B at FP16, batch sizes 1 / 8 / 32, measure tokens/sec and TTFT p50/p99.
Cold-start — measure time from "Container requested" to "first token out" for RunPod / Vast.ai / Lambda. The cold-start number matters most for batch / async workloads.
Reliability — 72-hour soak at 70% load, count pre-emptions and silent evictions.

5. CDN benchmarks

For CDN articles (Cloudflare, Bunny, Fastly, Akamai, CloudFront) we measure:

TTFB p50 / p95 from a probe in the target region (we run our own probes in 5 cities).
Cache hit ratio over 7 days against a fixed asset set (1 KB / 100 KB / 1 MB / 10 MB).
Real monthly bill at 1 TB, 10 TB, and 100 TB of egress, including request fees and image transformation charges.

6. Pricing math

Every price we publish has three numbers: first month, renewal month, and effective month at 36 months. We also separate out:

IPv4 add-on ($0 to $5/mo typical)
Backup fee (free to 30% of plan)
Bandwidth overage rate ($0.01 to $0.09/GB)
Setup fee (often waived, sometimes not)
Commitment period (monthly / annual / 2-yr / 3-yr)

7. Refresh cadence

Articles are tagged with a refresh tier. We re-test on this schedule:

Tier 1 (3 months): Best-of pages, "vs" comparisons, pricing pages, anything mentioning a current model (Llama, Qwen, Flux, Veo, H200).
Tier 2 (6 months): Single-vendor reviews, use-case guides.
Tier 3 (12 months): Evergreen reference material, history articles.

8. What we will not do

We will not change a recommendation because a vendor asked us to.
We will not publish a "10 best" listicle without running at least one of the 10 ourselves.
We will not copy a vendor's spec sheet and present it as a review.
We will not link to a provider we have not paid at least once.

9. Errata and corrections

If we get something wrong, we fix it at the top of the page and add a date stamp. Vendor reps are welcome to flag issues to our editorial team; corrections are made on the merits, not the relationship. We do not accept payment for corrections or reviews.

10. Where the raw data lives

All benchmark CSVs, command outputs, and screenshots are stored in our public archive:

GitHub: github.com/progeofde/benchmarks (placeholder — fill when repo exists)
Methodology doc on this site: /methodology.html