Is the RTX 3090 Still a Good Option for LLM Inference?

Mar 15, 2026

The Problem

I wanted to build a local LLM inference setup. My budget was around $3000, and I needed to run large language models like Llama 3.1 70B and Qwen 2.5 72B. I looked at the RTX 3090, RTX 4090, and RTX 5090, trying to figure out which GPU would give me the best value.

The RTX 5090 costs $3000-4000. The RTX 4090 costs around $2000-2200. Used RTX 3090s go for $600-850 on eBay. I was confused - surely the newer, more expensive cards must be significantly better for LLM inference?

I was wrong.

First Attempt: Looking at Raw Performance

I started by comparing raw specifications:

| Feature          | RTX 3090       | RTX 4090       | RTX 5090       |
|------------------|----------------|----------------|----------------|
| VRAM             | 24GB GDDR6X    | 24GB GDDR6X    | 24GB+          |
| Memory Bandwidth | ~1000 GB/s     | ~1008 GB/s     | Higher         |
| Architecture     | Ampere         | Ada Lovelace   | Latest         |
| Power Draw       | ~350W          | ~450W          | ~450W+         |

The specifications looked similar for VRAM. The 4090 and 5090 had newer architectures, but for inference, VRAM is what matters most. If a model doesn’t fit in memory, speed doesn’t matter.

I realized I needed to think differently about this problem.

Second Attempt: Cost Per VRAM

I calculated the cost per gigabyte of VRAM:

# Cost per GB of VRAM calculation

gpus = {
    "RTX 3090 (used)": {"price": 725, "vram": 24},  # Average $600-850
    "RTX 4090": {"price": 2100, "vram": 24},
    "RTX 5090": {"price": 3500, "vram": 24},  # Estimated
}

for gpu, specs in gpus.items():
    cost_per_gb = specs["price"] / specs["vram"]
    print(f"{gpu}: ${cost_per_gb:.2f}/GB VRAM")

When I ran this:

RTX 3090 (used): $30.21/GB VRAM
RTX 4090: $87.50/GB VRAM
RTX 5090: $145.83/GB VRAM

The numbers were clear. The RTX 3090 offers VRAM at nearly 5x better value than the 5090.

Third Attempt: Real-World Benchmarks

I dug into benchmarks from the r/LocalLLaMA community. Users reported running Qwen 3.5 122B across multiple 3090s:

4x RTX 3090s running Qwen 3.5 122B at 115 tps
Performance: 4x lower than 4090, 6x lower than 5090
Total VRAM: 96GB - can run models that single 4090/5090 cannot

The newer cards were faster. But here’s the key insight: a single 5090 with 24GB VRAM cannot load a 70B+ model at full precision. Four 3090s with 96GB total VRAM can.

For my $3000 budget, I could get:

One RTX 5090 (24GB VRAM)
One RTX 4090 (24GB VRAM)
Four to five RTX 3090s (96-120GB VRAM)

What I Learned

The decision depends on my use case:

For pure inference: RTX 3090 wins. I can run larger models by spreading across multiple GPUs.

For training/fine-tuning: RTX 4090 is better. The Ada Lovelace architecture has improvements for these workloads.

For image generation: RTX 4090 has measurable improvements over the 3090.

If I need warranty: Only 4090 and 5090 come with warranties. Used 3090s are sold as-is.

Multi-GPU Setup Example

If I go with multiple 3090s, here’s how I would configure inference:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model across multiple GPUs
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-72B-Instruct",
    device_map="auto",  # Automatically distributes across available GPUs
    torch_dtype=torch.float16,
    max_memory={i: "22GB" for i in range(4)}  # Reserve 2GB per GPU for overhead
)

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-72B-Instruct")

# Inference
inputs = tokenizer("Explain quantum computing", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_length=512)
print(tokenizer.decode(outputs[0]))

This setup lets me run a 72B model that would never fit on a single 24GB card.

The Risks of Buying Used

I researched the risks of buying used GPUs:

No warranty - If the card dies, I’m out the money
Mining cards - May have degraded VRAM or thermal issues
Power consumption - 4x 3090s draw ~1400W, need robust PSU

My mitigation strategy:

Buy from sellers with good return policies
Stress test immediately upon arrival
Use a quality 1600W+ power supply

Final Comparison

| Feature          | RTX 3090 (Used) | RTX 4090      | RTX 5090      |
|------------------|-----------------|---------------|---------------|
| Price            | $600-850        | $2000-2200    | $3000-4000    |
| VRAM             | 24GB GDDR6X     | 24GB GDDR6X   | 24GB+         |
| Inference Value  | 5/5 stars       | 3/5 stars     | 2/5 stars     |
| Training Value   | 3/5 stars       | 4/5 stars     | 5/5 stars     |
| Warranty         | None (used)     | Full          | Full          |
| Risk Level       | Medium-High     | Low           | Low           |

Summary

For my use case - running large language models for inference - the RTX 3090 is the best choice. With my $3000 budget, I can get four used 3090s with 96GB total VRAM. This lets me run models that a single 5090 cannot even load.

The newer cards are faster per-token, but that doesn’t matter if the model doesn’t fit in memory. For inference-focused workloads, VRAM capacity often beats raw compute speed.

If you’re doing training, fine-tuning, or image generation, consider the 4090. But for pure LLM inference, the 3090 offers unmatched value - assuming you’re comfortable with the risks of buying used hardware.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 r/LocalLLaMA Discussion
👨‍💻 NVIDIA RTX 3090 Specifications
👨‍💻 Hugging Face Transformers Documentation

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!