The Service Existed. Swarm Had Nowhere to Run It.

A service definition can be accepted by Docker Swarm even when the scheduler cannot place a task. During my browser-arcade homelab work, the distinction between scheduling and port delivery mattered. I do not have a complete original incident transcript, so the method below is documentation-based, not a fabricated replay.

Start with the task

Use read-only inspection on a manager:

docker service ls
docker service ps --no-trunc arcade-web
docker service inspect --pretty arcade-web
docker node ls

Read the full task error. If the service is pending, inspect eligibility, node availability, resource reservations, and image access before changing DNS or proxy configuration. On a manager, docker node ls shows node readiness and availability; docker node inspect --pretty <node-name> provides more detail. A node set to Drain or Pause cannot receive new tasks. A node that matches every label can still be ineligible if it cannot satisfy reserved CPU or memory. Do not confuse the service’s desired replica count with a successful placement. If the task starts and exits, examine startup and application logs. A task that never started may have no useful application logs.

Placement constraints

A constraint such as node.labels.arcade == true is a strict scheduling requirement. A service label does not satisfy a node label constraint. Inspect the intended node and the service’s placement requirements:

docker node inspect --format '{{json .Spec.Labels}}' <node-name>
docker service inspect --format '{{json .Spec.TaskTemplate.Placement}}' arcade-web

Constraints are mandatory; placement preferences are best-effort. Do not remove a constraint simply to obtain a green replica count. The service may need local storage, special hardware, or an intentionally limited host. Confirm the original reason for the constraint first.

Troubleshooting decision table

Observed state	Next evidence
Service exists, 0/1 replicas	`docker service ps --no-trunc`, constraints, node availability, reservations
Task rejected	Full task error and runtime configuration
Task started, then exited	Service logs and image/command/mount settings
Task running, direct HTTP fails	Target and published ports, publishing mode, firewall
Direct HTTP works, public URL fails	DNS, proxy upstream, TLS, ingress route

When a running task is still not reachable

Once Swarm shows the intended task running, switch from placement to delivery. First inspect the declared service ports:

docker service inspect --format '{{json .Endpoint.Spec.Ports}}' arcade-web
docker service ps --no-trunc arcade-web

An ingress-published port uses Swarm’s routing mesh, so a connection arriving on one Swarm node may be forwarded to a task on another. A host-published port bypasses that routing mesh and is exposed on the node where the task runs. Testing a different node in host mode is not a valid service-health check. Docker documents both modes, including the scheduling limitation of a fixed host-published port when multiple tasks share a node.

The published port and the application’s listening port are not interchangeable. For example, published=8080,target=80 declares an external port of 8080 and an internal target of 80. It does not prove the application listens on 80. Verify the actual service configuration, and test the expected endpoint from an authorized network before changing the reverse proxy.

A port conflict, firewall rule, stale proxy upstream, or wrong DNS answer can produce a failed browser request. None of them explains a task that never got scheduled. Keeping the investigation ordered prevents repairs in the wrong layer.

A private evidence record beats a public screenshot

Before modifying a service, collect the existing specification and the current task state. Store this information privately. The raw docker service inspect output may include environment variables, internal paths, image registry names, and other infrastructure details that do not belong in a public blog post.

Write down what you observed, which command produced it, what you changed, and what happened after the change. Keep inferred causes separate from actual task errors. A task error copied from an authorized environment is evidence; a plausible error invented after the fact is not.

Do not remove a placement constraint just to make a replica count green. It may enforce a storage or hardware requirement. If the task requires a bind-mounted directory, confirm the source path and expected files on the selected task node, not just the manager. A scheduler that moves the task to a different machine does not automatically move local storage.

Validate the fix and plan rollback

A practical final check follows the same boundaries as the failure investigation:

Confirm the desired replica count and eligible nodes.
Confirm the current task is running on the intended node.
Confirm any required local storage and permissions on that node.
Confirm the configured ingress or host publishing mode.
Test the direct application endpoint, then the reverse-proxy route.
Complete one real browser action rather than stopping at a successful page load.

Change one variable at a time. Where a service update supports it, docker service rollback arcade-web can revert the previous service specification. It cannot undo a separate node-label edit, reverse a DNS change, or restore deleted files. Record those changes and their own recovery steps independently.

The final user action matters. As with Reachable Is Not the Same as Playable, a reachable page is not proof of a usable application. And as with The Build Passed. Production Did Not Move., success in one stage does not guarantee success in the next.

Sources and further reading

AI transparency

AI assisted with research and editing. The original incident transcript is incomplete; no specific task errors or unperformed test results are claimed. The commands are documented inspection methods, not a claim that they were executed against a live Swarm during this editorial review.