The big picture: control plane & Pods

Architecture & Pods — run the reconcile loop by hand

Mental model: you never tell Kubernetes to start a container. You declare the desired
state and the cluster reconciles reality toward it, forever. One loop, one writer:
  kubectl  ->  API server  ->  etcd        API server is the single front door and the
                                           only component that reads/writes state; it
                                           persists that state in etcd.
  scheduler                                watches for a Pod with no Node yet and picks
                                           the best Node. If none fits, the Pod stays Pending.
  kubelet (on the chosen Node)             pulls the image, starts the containers, reports
                                           real status back up to the API server.
  controllers                              never stop comparing actual vs desired; delete a
                                           Pod and the Deployment's controller recreates it.
                                           That constant steering is self-healing.
No component commands another directly — they all watch the API server's truth and reconcile.

1 · Hand desired state to the API server
    kubectl create deployment web --image=nginx
    # or, from a manifest:
    kubectl apply -f deploy.yaml
    Either way the Deployment's controller owns the Pod; you never create the Pod yourself.

2 · Watch it come up  (-o wide shows NODE + IP)
    kubectl get pods -o wide
      NAME           READY  STATUS   RESTARTS  AGE  IP        NODE                        ...
      web-...        1/1    Running  0         ...  10.244..  sprkd-verify-control-plane  ...
    A healthy Pod reads Running with a real IP and a real NODE.

3 · Break scheduling on purpose (a Pod the scheduler can't place)
    kubectl get pods -o wide
      NAME           READY  STATUS    RESTARTS  AGE  IP       NODE     NOMINATED NODE  READINESS GATES
      pending-demo   0/1    Pending   0         5s   <none>   <none>   <none>          <none>
    STATUS Pending with IP <none> and NODE <none> = the scheduler couldn't place it.
    This is NOT an app crash — the container never started anywhere.

4 · Read the Events — the cluster narrates what happened
    kubectl describe pod pending-demo
      Events:
        Type     Reason            Age  From               Message
        ----     ------            ---- ----               -------
        Warning  FailedScheduling  5s   default-scheduler  0/1 nodes are available: 1 node(s)
          had untolerated taint(s). no new claims to deallocate, preemption: 0/1 nodes are
          available: 1 Preemption is not helpful for scheduling.
    Reason FailedScheduling from the default-scheduler, cause: an untolerated taint.
    The fix follows the message — satisfy the taint, don't restart the Pod.

5 · Fix it (two documented ways)
    (a) Let the Pod tolerate the taint — add to the Pod spec:
          tolerations:
          - key: <key>
            operator: "Exists"
    (b) Or remove the taint from the node so anything can schedule (note the trailing dash):
          kubectl taint nodes <node> <key>-

6 · See self-healing — the controller, not you, replaces Pods
    kubectl delete pod web-...
    kubectl get pods -o wide      # the Deployment's controller has already recreated it
    You deleted a Pod; the controller noticed the gap between actual and desired and
    reconciled it back. That is the loop, working.

What each part does
    API server   single front door; only reader/writer of cluster state (persists to etcd)
    etcd         where cluster state actually lives
    scheduler    assigns an unscheduled Pod to a Node; can leave it Pending
    kubelet      per-Node agent; pulls image, runs containers, reports status
    controllers  compare desired vs actual and drive the gap to zero (self-healing)

Gotchas
    - Pending with NODE <none> is a scheduling problem, not a crashing app — read Events, not logs.
    - Restarting/deleting a Pending Pod won't help; the same taint blocks the replacement.
    - Deleting a Deployment-owned Pod does not "stop" it — the controller recreates it.

Verify:  kubectl describe pod <pod> and read the Events section at the bottom — Events turn a
         silent Pending into a named, fixable cause (here: FailedScheduling / untolerated taint).