Redis on OCI —
The iptables Fix

📅 Apr 2025 ⏱ 11 min read 🏷

Redis was running. Auth was working. The OCI Security List was open. The iptables rule existed. And yet — ConnectionTimeoutError. Every fix I tried changed nothing. This is the story of how a single line position in iptables blocked everything, and how tcpdump found it.

Everything That Was Already Working

This is the frustrating part. Most of the setup was correct. The bug was invisible until the right tool was used.

Redis Was Running and Responding

Local connection worked perfectly. redis-cli -a "password@redis" ping returned PONG. Auth was configured correctly in redis.conf with requirepass redis@vyshnav. The service was up.

Redis Was Bound to All Interfaces

bind 0.0.0.0 was set in /etc/redis/redis.conf and protected-mode no was enabled. ss -tlnp | grep 6379 confirmed Redis was listening on 0.0.0.0:6379 — not just localhost.

OCI Security List Was Open

The VCN Security List had a Stateful Ingress rule: TCP, Source 0.0.0.0/0, Destination Port 6379. The OCI Console showed green. The rule was there.

iptables Had an ACCEPT Rule

sudo iptables -L INPUT -n --line-numbers showed an ACCEPT rule for port 6379. It existed. It looked correct. This is what made the bug so confusing — the rule was there, but it wasn't working.

The Four Layers Between Client and Redis

Before debugging anything, it helps to understand every hop a packet takes on its way to Redis. Each layer can silently drop traffic.

Phase 01
☁️ OCI Security List

Oracle's virtual firewall at the VCN (network) level. Ingress rules must explicitly allow TCP on port 6379. Managed in the OCI Console under Networking → VCN → Subnet → Security Lists.

OCI Console
Phase 02
🛡️ OCI NSG (Network Security Group)

A secondary security layer attached to the VM's VNIC. If your instance has an NSG attached, traffic must pass its rules too — even if the Security List allows it. Easy to forget.

OCI NSG
Phase 03
🔥 iptables (OS Firewall)

Linux kernel-level packet filtering. Rules are evaluated top to bottom — first match wins. OCI Ubuntu images ship with a default REJECT rule that catches everything not explicitly allowed above it.

Linux kernel
Phase 04
🍃 Redis bind / protected-mode

Redis itself must be configured to listen on 0.0.0.0 (all interfaces) and have protected-mode no set when running with auth. If Redis only binds to 127.0.0.1, external connections are refused at the application level.

redis.conf

When an external client connects to Redis on an OCI VM, the packet travels through four distinct layers — and every single one of them can block it without giving you a clear error message.

TL;DR

text
Debugging checklist for Redis on OCI:

  1. Redis config       → bind 0.0.0.0, protected-mode no, requirepass set
  2. Listening check    → ss -tlnp | grep 6379  (must show 0.0.0.0)
  3. OCI Security List  → Ingress TCP 6379 open in VCN subnet
  4. OCI NSG            → check if instance has NSG attached
  5. iptables order     → ACCEPT rule must be ABOVE REJECT rule
  6. tcpdump            → if packets arrive but no response: iptables
  7. Persist rules      → netfilter-persistent save
  8. Test externally    → never test from the same VM (hairpin NAT)

  Root cause was:  iptables -A  (append = after REJECT)
  Fix was:         iptables -I INPUT 1  (insert = before REJECT)

Three hours of debugging, every config was correct, and the problem was a single keyword — -A instead of -I when adding an iptables rule.

Decoding the Error Messages

The two errors pointed to different problems — understanding what each actually means was the first step to finding the real cause.

Error What It Actually Means Where to Look
NOAUTH Connected but no password provided Client config — add auth
WRONGPASS Connected but wrong password Client config — check password
ConnectionTimeoutError Packet reached server but no response came back iptables, NSG — firewall dropping response
No route to host Packet was actively rejected (ICMP unreachable sent back) iptables REJECT rule — packet matched it
Connection refused Port not open on that machine Redis not running or wrong port

ConnectionTimeoutError means packets are arriving but nothing is coming back. The server received the SYN packet but the firewall dropped the response. No route to host is actually more informative — it means the packet hit a REJECT rule and an ICMP "host unreachable" was sent back. That ICMP response is the clue.

The Root Cause — iptables Rule Order

iptables evaluates rules top to bottom and stops at the first match. The ACCEPT rule for port 6379 existed — but it was positioned after a REJECT ALL rule. The traffic never reached it.

bash
# What the iptables chain looked like (simplified)
sudo iptables -L INPUT -n --line-numbers

# num  target   prot  source    destination
# 1    ACCEPT   tcp   0.0.0.0   0.0.0.0     dpt:22    ← SSH allowed
# 2    ACCEPT   tcp   0.0.0.0   0.0.0.0     dpt:5432  ← PostgreSQL allowed
# 3    ACCEPT   all   0.0.0.0   0.0.0.0     state RELATED,ESTABLISHED
# 4    REJECT   all   0.0.0.0   0.0.0.0                ← BLOCKS EVERYTHING ❌
# 5    ACCEPT   tcp   0.0.0.0   0.0.0.0     dpt:6379  ← too late, never reached

# Redis traffic hits rule 4 → REJECTED
# Rule 5 never evaluated

iptables First Match Wins

Rules are evaluated sequentially from top to bottom. The first matching rule wins and processing stops. A REJECT rule above your ACCEPT rule means the ACCEPT rule is dead code — it will never be reached by any packet that also matches the REJECT.

OCI Ubuntu images come with iptables pre-configured for security. The default ruleset includes a REJECT all rule near the bottom of the INPUT chain. When the port 6379 ACCEPT rule was added (likely with -A, which appends to the end), it was placed after the REJECT rule.

Every incoming Redis packet matched the REJECT rule first and was dropped. The ACCEPT rule was never reached. The rule existed. The logs showed it. But its position made it completely ineffective.

Complete Redis Configuration

The full redis.conf changes needed to allow external connections securely.

bash
# /etc/redis/redis.conf — required changes

# 1. Bind to all interfaces (not just localhost)
bind 0.0.0.0

# 2. Disable protected mode (required when using password auth)
protected-mode no

# 3. Set a strong password
requirepass password@redis

# 4. Limit memory on low-RAM systems
maxmemory 100mb
maxmemory-policy allkeys-lru

# Apply changes
sudo systemctl restart redis
sudo systemctl status redis

# Verify listening on all interfaces
ss -tlnp | grep 6379
# → 0.0.0.0:6379  ✅  (not 127.0.0.1:6379)

# Connect from external machine
redis-cli -h redis.yourdomain.com -p 6379 -a "password@redis"
# Useful commands once connected:
# KEYS *              → list all keys
# GET keyname         → string value
# HGETALL keyname     → hash fields
# LRANGE keyname 0 -1 → list items
# INFO memory         → memory usage

The Fix

Two commands. Insert the ACCEPT rules at the top of the chain so they're evaluated before the REJECT rule, then persist them across reboots.

bash
# -I inserts at position 1 (top of chain)
# -A would append to the end — wrong
sudo iptables -I INPUT 1 -p tcp --dport 6379 -j ACCEPT
sudo iptables -I INPUT 1 -p tcp --dport 6380 -j ACCEPT

# Verify — rule 1 and 2 should now be your ACCEPT rules
sudo iptables -L INPUT -n --line-numbers

# Persist across reboots — without this, rules reset on restart
sudo apt install -y iptables-persistent
sudo netfilter-persistent save

# Test from external machine
redis-cli -h redis.yourdomain.com -p 6379 -a "password@redis" ping
# → PONG ✅
⚠️

OCI hairpin NAT: You cannot connect to your VM's public IP from inside the same VM. OCI doesn't support hairpin NAT. Always test Redis connectivity from an external machine — testing from the VM itself via its public IP will always fail regardless of firewall rules.

The difference between -A (append) and -I (insert) is the entire bug. -A always adds to the end of the chain — after any existing REJECT rules. -I INPUT 1 inserts at position 1, before everything else.

Always use netfilter-persistent save after changing iptables rules. Without it, every reboot wipes your rules and the connection fails again. The iptables-persistent package writes rules to /etc/iptables/rules.v4 and restores them on boot.

The Breakthrough — tcpdump

Every configuration check said things should work. The real answer came from watching actual packets instead of reading config files.

bash
# On the VM — watch for any traffic on port 6379
sudo tcpdump -i any port 6379

# From external machine — attempt connection
redis-cli -h host.youdomain.com -p 6379 -a "password@redis" ping

# tcpdump output showed:
# 14:22:01 IP external.ip.here > vm.ip.here.6379: Flags [S]  ← SYN arrived ✅
# (silence — no SYN-ACK sent back)                          ← response dropped ❌

# This confirmed: packets reach the VM but iptables kills the response
# OCI Security List ✅ — NSG ✅ — Redis config ✅ — iptables ❌
🔍

Debug principle: When config looks correct but connections fail, watch the actual packets. tcpdump shows you ground truth — it bypasses every assumption you've made about what should be happening.

The key insight: stop reading config and start watching traffic. tcpdump captures packets at the network interface level — before iptables drops them. If packets are arriving but nothing goes back, you've confirmed the problem is iptables, not OCI, not DNS, not the client.

Running tcpdump while a connection attempt was made from an external machine showed incoming SYN packets arriving at the VM. The VM received them. But no SYN-ACK was sent back. The packets were dying somewhere inside the VM — and the only thing that could do that was iptables.

Security Considerations

Redis exposed publicly with only a password is a risk. Here's what to do about it in different environments.

Always Use a Strong Password

Redis has no rate limiting on auth attempts. A weak password can be brute-forced quickly. Use a long random string — at least 32 characters. openssl rand -base64 32 generates a good one.

Restrict by IP in iptables

Instead of accepting all IPs on port 6379, restrict to specific trusted IPs: sudo iptables -I INPUT 1 -p tcp --dport 6379 -s YOUR_IP -j ACCEPT. This means even if someone guesses your password, they can't reach the port at all.

Use SSH Tunnel for Local Development

Never expose Redis publicly just for local development access. Use SSH tunneling instead: ssh -L 6379:localhost:6379 user@vm-ip. Then connect to localhost:6379 on your machine — the connection is encrypted and Redis never needs to be publicly exposed.

Private Access in Production

In production, Redis should be on a private subnet accessible only to your application servers. Use OCI's private IP addresses for inter-service communication. Public exposure should be a last resort, not the default.

A real debugging journey — written so you don't spend 3 hours on the same problem.

Redis OCI iptables Linux DevOps networking debugging