Server Down? Phone-Only Incident Response over SSH

When your server goes down and you only have a phone, confirm the outage, check host health, inspect service logs, make one justified recovery change, and verify the public service. Use this ordered SSH runbook to contain the incident, with clear limits for when to hand off.

What should I check first when my server goes down?

First confirm impact, second check the host, third inspect the affected service. Only then choose a recovery action. This order prevents a weak cellular connection from looking like a server failure and a full disk from becoming an endless restart loop.

1. Confirm impact: open your external monitoring dashboard and the affected URL. Check the alert’s host, environment, start time, and failing check. Look at the latest deployment and provider status too.
2. Reach and identify the host: establish SSH, then verify where you landed before using elevated privileges.
3. Check resource pressure: look for exhausted disk space, inodes, or memory before asking a process to start again.
4. Read service evidence: inspect status and a bounded slice of logs around the failure.
5. Make one justified change: restart the affected component or trigger a documented rollback, then verify through the public path.

If unrelated websites also fail on your phone, try a usable Wi-Fi or cellular connection before diagnosing the server. Your phone’s browser is one observation; an independent external monitor helps distinguish your connection from customer impact. A failed ping alone cannot settle that question.

Acknowledge the incident through your normal channel. Tell the team you are responding from a phone, and record actions as you go so another responder does not unknowingly make competing changes.

Command scope: the host examples below target Ubuntu 22.04/24.04 or Debian 12 using systemd, with standard GNU/procps utilities. Nginx examples assume a packaged installation managed as nginx.service. Docker commands are a separate branch for an existing Docker Engine host. Run commands on the remote server, one block at a time; do not paste the entire article as a script.

How do I reach my server from a phone during an outage?

Use your established SSH address, port, username, and key. Verify an unfamiliar host fingerprint against a trusted record; an outage is no reason to dismiss a changed-key warning. If access is not already configured, use your provider’s console or contact someone with working access rather than changing firewall rules blindly. See SSH lockout recovery for that branch.

This is where TermAI fits the incident: on iOS or Android, use its SSH connection to run the checks below. If the server is already reachable through your authorized tailnet, built-in Tailscale provides the private network route. The server still needs working SSH and valid authentication; neither tool revives an offline machine.

For a command you cannot recall, ask the AI assistant something specific: “Ubuntu 24.04, nginx.service: suggest a read-only command showing the latest 40 journal entries, without a pager.” Review the proposed host-independent syntax, service name, and flags, then confirm execution. TermAI suggests commands you approve; it is not an automatically executing incident agent. Avoid sending secrets or unredacted customer logs. Its SFTP support can retrieve an existing diagnostic file you have permission to read for a handoff. The free tier includes five AI calls per day and unlimited SSH/SFTP, so keep this runbook available without depending on AI access.

After connecting, identify the machine:

hostname

whoami

date -u

Compare the hostname with the alert and provider inventory. The UTC timestamp anchors your incident notes. If the machine or account is wrong, stop here.

How do I check whether the server is out of resources?

Get a short host snapshot before touching the service:

uptime

free -h

df -h / /var

df -i / /var

Unexpectedly short uptime suggests a recent boot worth investigating. Load average needs context about CPU count and normal workload; it is not a CPU percentage. For memory, examine available, not just free, because reclaimable cache affects what applications can use. See the Ubuntu free manual.

The disk checks cover the filesystems containing / and /var; add your actual application or database data path if it lives elsewhere. Repeated rows can simply mean both paths share one filesystem. Block space and inodes are separate limits: either can prevent new files. The flags are documented in the Ubuntu df manual.

If storage is exhausted, branch here. Use your established storage or log-retention runbook, or escalate to someone who can identify safe cleanup targets. Do not delete database files, erase incident logs, or broadly prune Docker resources to make a restart fit. If the shell itself stalls or resource pressure is severe, use monitoring and provider-console evidence; adding heavy recursive scans can make diagnosis harder.

How do I check service status and read logs on my phone?

For the Nginx example, inspect the exact unit:

sudo systemctl status nginx.service --no-pager --full --lines=10

An inactive or failed service can make this command return a nonzero exit status; that is useful diagnostic information. --full avoids ellipsizing status text, while --no-pager avoids an interactive pager on the small screen. These options are covered by the systemctl manual.

Then retrieve recent journal entries:

sudo journalctl -u nginx.service --since "15 minutes ago" -n 40 --no-pager -o short-iso

The time window and line count are viewing limits, not incident thresholds. Adjust the window to include the alert’s start time. Begin without a severity filter: startup messages and dependency failures may explain the event even when they are not classified as errors. See the journalctl manual.

Nginx request errors may be in a file instead of the journal. For the standard packaged log path, if that is where your configuration writes:

sudo tail -n 40 /var/log/nginx/error.log

Look for the earliest relevant failure, then distinguish a configuration error, an unavailable upstream, and resource exhaustion. If Nginx is active but reports upstream connection failures, inspect the application unit next using its actual name. Repeatedly restarting the proxy will not repair a failed application or database dependency.

For Docker instead of a host service, list names and status, including stopped containers:

sudo docker ps -a --format '{{.Names}}: {{.Status}}'

Use the exact container name from that output. The next example assumes it is web; replace that name before running:

sudo docker logs --since 15m --tail 40 --timestamps web

The compact listing and bounded logs use documented Docker listing and logging options. Logging configuration determines what is available; empty output does not prove the application is healthy. For rootless Docker, use the account and Docker context that own the workload rather than these sudo examples.

How do I safely restart a service from my phone?

Restart only when evidence identifies the component and your runbook permits it. Check whether another responder or deployment is already working on it. A syntax error, full disk, or failing dependency needs its own recovery path first.

For packaged Nginx, validate the on-disk configuration:

sudo nginx -t

This checks syntax and attempts to open referenced files; it does not test application health. If validation fails, stop and use the approved configuration rollback. The behavior is documented by Nginx.

If Nginx is stopped or failed and validation passes, start it:

sudo systemctl start nginx.service

If it is running but your diagnosis and runbook call for a restart, this is the alternative, not the next automatic step:

sudo systemctl restart nginx.service

A restart interrupts the service. For a running Nginx instance that only needs an approved configuration change loaded, use a validated reload instead:

sudo systemctl reload nginx.service

For a confirmed standalone Docker container named web, the equivalent targeted recovery is:

sudo docker restart web

Docker can forcibly terminate the container if it does not stop within its timeout, so this is not a harmless probe. See Docker restart behavior. For orchestrated workloads, use the orchestrator’s established recovery workflow instead of fighting its desired state with manual container restarts.

If the failure immediately returns, do not keep bouncing it. Capture the new error and escalate or roll back the implicated change. Avoid restarting the whole host, SSH daemon, or network stack as a substitute for identifying the failed component.

How do I know the website is actually back up?

Check service status again, then repeat the bounded log query. A running process alone does not prove requests reach the application.

Use your known, non-mutating health URL. This example assumes the application exposes /health on loopback port 8080; replace the URL with the endpoint in your runbook:

curl --noproxy '*' -sS --connect-timeout 3 --max-time 10 -o /dev/null -w 'HTTP %{http_code}\n' http://127.0.0.1:8080/health

This bounds the request and prints the HTTP status without downloading a body to the screen. It still shows connection errors. --noproxy '*' ensures the loopback check bypasses configured proxies. Curl’s timeout and output behavior are documented in its manual.

Compare the response with your endpoint’s documented expected result. An HTTP error can still arrive through a successful curl transfer; the printed status matters. If the health contract requires particular response content, inspect that too rather than relying on the status alone.

Now open the public URL in the phone browser and check external monitoring. A healthy loopback endpoint with a failing public path points toward another layer, such as the reverse proxy, load balancer, DNS, TLS, or routing. Do not restart the healthy application merely because the external path still fails.

Observe recovery through your normal monitoring interval and check for renewed errors. Record the action, time, and observed result. Describe the service as mitigated while the underlying cause remains unresolved.

What if SSH disconnects halfway through a command?

A dropped connection means the result is unknown. A restart request may already have reached systemd or Docker. Reconnect and check status, recent logs, and the endpoint before repeating a write operation.

For longer shell work, use an already-installed server-side tmux session before starting:

tmux new -s incident

After reconnecting to the same host as the same user:

tmux attach -t incident

If that session already exists at the outset, attach instead of creating it again. tmux preserves its terminal session across SSH disconnections; it does not survive a host reboot, guarantee a deployment succeeds, or turn a half-entered command into a completed one. See the tmux getting-started guide.

Keep the SSH app in the foreground while actively working, but assume phone locking or network changes can break the connection. Do not depend on keepalives to make commands durable. If tmux is unavailable, stick to short diagnostic calls and established server-side jobs with separately inspectable status; installing new tooling during an outage is not a prerequisite for triage.

How do I avoid command mistakes on a phone keyboard?

Keep a plain-text runbook accessible on the phone with exact host names, service names, health URLs, and approved recovery steps. Prepare and test it over cellular before an incident. Do not store private keys or passwords in that note. For access preparation, see mobile SSH production safety.

When should I stop troubleshooting from my phone?

A phone is enough when you can identify the target, understand a bounded action, and verify its result. It becomes inadequate when the decision requires context you cannot reliably inspect.

• Database repair or uncertain migrations: bring in the database owner with a laptop. Preserve evidence and use the documented recovery process; do not improvise rollback SQL.
• A bad deployment: trigger a known rollback through the existing deployment system, with the approved artifact and visible job status. Check schema compatibility first. Avoid editing source, building a new release, or deploying an unreviewed fix from the emergency shell.
• Broad configuration or network changes: hand off when you need large diffs, multiple hosts, or a change that could remove your only access path.
• Suspected compromise: engage your security incident process. Restarting or deleting files can destroy evidence while leaving the intrusion unresolved.
• No stable connection or usable console: contact the secondary responder or provider support. A phone cannot supply an out-of-band recovery path that was never configured.

Give the next responder the affected host and environment, alert start time, last relevant change, commands actually issued, their observed results, and the next hypothesis. Mark any operation interrupted by a disconnect as outcome unconfirmed. Stay available until someone explicitly takes ownership.

What else should I know about phone-only incident response?

Can I fix a server outage using only my phone?

You can check host health, inspect logs, restart a known service, and trigger an established rollback. Uncertain database recovery, security incidents, and changes requiring large diffs need a laptop or another responder.

Should I reboot the server when my website goes down?

Not as the first step. Confirm the outage, inspect resources and service logs, and recover the smallest affected component. Use a host reboot only when the incident runbook calls for it and recovery access is available.

Does an SSH disconnect mean my restart failed?

No. The request may already have reached the service manager. Reconnect, inspect service status and logs, and check the public endpoint before issuing another restart.

Can I deploy a fix from my phone?

Use an existing deployment or rollback workflow with a known artifact, visible job status, and a verified recovery path. Avoid improvising production edits or migrations in a mobile shell.