This is the most honest article in the series. Every tutorial makes things look easy. You follow the steps, it works, done. Reality is messier.<\/p>
Building this home lab I hit 15 distinct errors. Some took minutes to fix. Some took hours. All of them taught me something I would not have learned from reading documentation.<\/p>
I am documenting every one with the exact error message, what caused it, and exactly how I fixed it.<\/p>
Error:<\/p>
[ERROR]: the role 'common' was not found in ...\/playbooks\/roles<\/code><\/pre>Cause: Ansible looked for roles relative to the playbook file location (playbooks\/) instead of the project root. It looked in playbooks\/roles\/ which does not exist.<\/p>
Fix: Create ansible.cfg at the project root:<\/p>
[defaults]\nroles_path = roles\ninventory = inventory\/hosts.ini\nremote_user = reza\nhost_key_checking = False<\/code><\/pre>Lesson: Always create ansible.cfg. Do not rely on Ansible’s default path behavior.<\/p>
Error 2 – Sudo Requires Interactive Authentication<\/h3>
Error:<\/p>
sudo: interactive authentication is required<\/code><\/pre>Cause: Ubuntu’s default sudoers configuration requires a password for sudo. Ansible cannot handle interactive prompts.<\/p>
Fix: SSH into each server and run:<\/p>
echo \"reza ALL=(ALL) NOPASSWD:ALL\" | sudo tee \/etc\/sudoers.d\/reza<\/code><\/pre>Lesson: Set up passwordless sudo before running any Ansible playbooks on a fresh Ubuntu install.<\/p>
Error 3 – Variables Undefined<\/h3>
Error:<\/p>
'node_name' is undefined\n'timezone' is undefined<\/code><\/pre>Cause: Ansible loads group_vars\/ and host_vars\/ relative to the inventory file location. When the inventory is at inventory\/hosts.ini but Ansible is looking relative to the playbook at playbooks\/, it cannot find the vars directories.<\/p>
Fix: Move all variables directly into [all:vars] inside inventory\/hosts.ini as simple key=value pairs:<\/p>
[control_plane]\njarvis ansible_host=10.0.0.148 node_name=jarvis node_role=control-plane\n\n[all:vars]\ntimezone=America\/Vancouver\nlab_user=reza<\/code><\/pre>Lesson: For simple Ansible projects, put everything in [all:vars]. Do not rely on group_vars\/ and host_vars\/ with .ini inventories.<\/p>
Error 4 – apt-key Removed in Ubuntu 24.04<\/h3>
Error:<\/p>
Failed to find required executable \"apt-key\"<\/code><\/pre>Cause: Ubuntu 24.04 removed apt-key entirely. It was deprecated in 22.04 and removed completely in 24.04.<\/p>
Fix: Use the modern GPG keyring method:<\/p>
curl -fsSL https:\/\/download.docker.com\/linux\/ubuntu\/gpg \\\n | gpg --dearmor -o \/etc\/apt\/keyrings\/docker.gpg\nchmod a+r \/etc\/apt\/keyrings\/docker.gpg<\/code><\/pre>Repo line:<\/p>
deb [arch=amd64 signed-by=\/etc\/apt\/keyrings\/docker.gpg] https:\/\/download.docker.com\/linux\/ubuntu jammy stable<\/code><\/pre>Lesson: Always check for Ubuntu version-specific breaking changes before assuming a tutorial will work on Ubuntu 24.04.<\/p>
Error 5 – .kube Directory Does Not Exist<\/h3>
Error:<\/p>
Destination directory \/home\/reza\/.kube does not exist<\/code><\/pre>Cause: Ansible tried to copy the kubeconfig file before creating the directory. Wrong task order.<\/p>
Fix: Create the directory first, then copy:<\/p>
- name: Create .kube dir\n file:\n path: \/home\/{{ lab_user }}\/.kube\n state: directory\n\n- name: Copy kubeconfig\n copy:\n src: \/etc\/rancher\/k3s\/k3s.yaml\n dest: \/home\/{{ lab_user }}\/.kube\/config\n remote_src: yes<\/code><\/pre>Lesson: Always create parent directories before writing files. Task order is execution order in Ansible.<\/p>
Error 6 – Loki Requires Object Storage Backend<\/h3>
Error:<\/p>
Cannot run scalable targets without an object storage backend<\/code><\/pre>Cause: The Loki Helm chart changed its default deployment mode. Newer versions default to distributed\/scalable mode which requires S3 or GCS.<\/p>
Fix: Force single binary mode:<\/p>
helm install loki grafana\/loki \\\n --set deploymentMode=SingleBinary \\\n --set loki.auth_enabled=false \\\n --set loki.storage.type=filesystem \\\n --set singleBinary.replicas=1 \\\n --set read.replicas=0 \\\n --set write.replicas=0 \\\n --set backend.replicas=0 \\\n --set loki.useTestSchema=true<\/code><\/pre>Lesson: Helm chart defaults change between versions. Always pin chart versions with –version x.y.z.<\/p>
Error 7 – Loki Requires Schema Config<\/h3>
Error:<\/p>
You must provide a schema_config for Loki<\/code><\/pre>Cause: Loki v3.x requires explicit schema configuration. Older versions had built-in defaults.<\/p>
Fix: Add –set loki.useTestSchema=true. The error message told us exactly what to do.<\/p>
Lesson: Read error messages fully before searching for solutions. The fix was in the error itself.<\/p>
Error 8 – Jenkins adminPassword Renamed<\/h3>
Error:<\/p>
controller.adminPassword no longer exists.\nIt has been renamed to controller.admin.password<\/code><\/pre>Cause: The Jenkins Helm chart renamed this parameter in a recent version.<\/p>
Fix:<\/p>
# Old (broken)\n--set controller.adminPassword=homelab123\n\n# New (correct)\n--set controller.admin.password=homelab123<\/code><\/pre>Lesson: Helm chart parameter names change between versions. Read the error message fully, it usually tells you the new name.<\/p>
Error 9 – Route 53 Private Hosted Zone Requires VPC<\/h3>
Error:<\/p>
When creating a private hosted zone you must specify VPCId and VPCRegion<\/code><\/pre>Cause: Route 53 private hosted zones must be associated with an AWS VPC. Home lab servers are not in AWS.<\/p>
Fix: Use a public hosted zone instead. Records point to private IPs (10.0.0.x) which are unreachable from outside the home network:<\/p>
aws route53 create-hosted-zone \\\n --name rezaops.com \\\n --hosted-zone-config Comment=\"Homelab DNS\"\n # removed: PrivateZone=true<\/code><\/pre>Lesson: AWS private hosted zones are for VPC-internal DNS only. Not usable for bare metal home labs.<\/p>
Error 10 – Ansible Loop Cannot Parse List from .ini Inventory<\/h3>
Error:<\/p>
The loop value must resolve to a 'list', not 'str'<\/code><\/pre>Cause: .ini format inventory files read every value as a plain string. JSON embedded in a string is still a string. Ansible cannot loop over it.<\/p>
Fix: Avoid loops. Use a single AWS CLI batch call instead:<\/p>
aws route53 change-resource-record-sets \\\n --change-batch '{\"Changes\":[\n {\"Action\":\"UPSERT\",\"ResourceRecordSet\":{\"Name\":\"grafana.rezaops.com\",...}},\n {\"Action\":\"UPSERT\",\"ResourceRecordSet\":{\"Name\":\"jenkins.rezaops.com\",...}}\n ]}'<\/code><\/pre>Lesson: .ini inventory files only support simple string values. Use YAML inventory format if you need lists or dicts.<\/p>
Error 11 – Pi-hole v6 Completely Broken for Custom DNS<\/h3>
What happened: Pi-hole v6 replaced dnsmasq with its own FTL DNS engine. As a result:<\/p>
- \/etc\/dnsmasq.d\/ config files are ignored<\/li>
- pihole restartdns returns: local: FTL_PID_FILE: readonly variable<\/li>
- pihole reloaddns partially broken<\/li><\/ul>
Fix: Replace Pi-hole entirely with CoreDNS running as a K8s pod:<\/p>
rezaops.com {\n hosts {\n 10.0.0.200 grafana.rezaops.com\n 10.0.0.200 jenkins.rezaops.com\n fallthrough\n }\n}\n. {\n forward . 1.1.1.1 8.8.8.8\n cache 30\n}<\/code><\/pre>Lesson: Pi-hole v6 was a major breaking release. CoreDNS is a better fit for a Kubernetes-based home lab anyway.<\/p>
Error 12 – MetalLB Cannot Assign IP to Mixed TCP\/UDP Service<\/h3>
What happened: Deployed CoreDNS with a single Service handling both TCP and UDP on port 53. The service stayed pending forever.<\/p>
Fix: Split into two services with the same allow-shared-ip annotation:<\/p>
# Both services get the same IP\nmetadata:\n annotations:\n metallb.universe.tf\/allow-shared-ip: \"coredns-dns\"<\/code><\/pre>One service for TCP port 53, one for UDP port 53. MetalLB assigns the same IP to both.<\/p>
Lesson: MetalLB cannot handle mixed-protocol services in L2 mode. When a LoadBalancer service stays pending, check MetalLB logs.<\/p>
Error 13 – Homepage ConfigMap Not Found<\/h3>
Error:<\/p>
MountVolume.SetUp failed: configmap \"homepage-html\" not found<\/code><\/pre>Cause: Ansible tried to create the ConfigMap before copying the HTML file to the remote server. Wrong task order.<\/p>
Fix: Copy the file first, then create the ConfigMap:<\/p>
- name: Copy HTML to remote # first\n copy:\n src: index.html\n dest: \/tmp\/homepage-index.html\n\n- name: Create ConfigMap # second\n shell: kubectl create configmap homepage-html --from-file=index.html=\/tmp\/homepage-index.html ...<\/code><\/pre>Lesson: Same root cause as Error 5. Task order matters in Ansible. Dependencies must exist before the tasks that use them.<\/p>
Error 14 – K3s Registries Directory Missing on Worker<\/h3>
Error:<\/p>
Destination directory \/etc\/rancher\/k3s does not exist (on thor)<\/code><\/pre>Cause: K3s agent does not create \/etc\/rancher\/k3s\/ automatically on worker nodes. Only the server does.<\/p>
Fix: Create the directory before writing the file:<\/p>
- name: Create K3s config directory\n file:\n path: \/etc\/rancher\/k3s\n state: directory\n mode: '0755'\n\n- name: Write registries.yaml\n copy:\n dest: \/etc\/rancher\/k3s\/registries.yaml\n ...<\/code><\/pre>Lesson: Never assume a directory exists on a remote machine. Always use file: state: directory before writing files.<\/p>
Error 15 – Registry IP Mismatch<\/h3>
What happened: Tried to push an image to 10.0.0.200:5000 and got connection timeout.<\/p>
Cause: MetalLB assigned the registry service a different IP than expected. I assumed it would get 10.0.0.200 but it got 10.0.0.206. MetalLB assigns IPs sequentially based on the order services are created.<\/p>
Fix: Always check the actual assigned IP before using:<\/p>
kubectl get svc -n registry\n# Check EXTERNAL-IP column, use that value<\/code><\/pre>Update inventory:<\/p>
registry_ip=10.0.0.206<\/code><\/pre>Lesson: Never assume which IP MetalLB assigns. Always kubectl get svc to confirm.<\/p>
Summary<\/h3>- Error 1, Ansible role not found, missing ansible.cfg<\/li>
- Error 2, Sudo requires password, no NOPASSWD in sudoers<\/li>
- Error 3, Variables undefined, group_vars not loading in .ini inventory<\/li>
- Error 4, apt-key removed, Ubuntu 24.04 breaking change<\/li>
- Error 5, .kube dir missing, wrong task order<\/li>
- Error 6, Loki needs object storage, Helm chart default changed<\/li>
- Error 7, Loki needs schema config, Loki v3 requirement<\/li>
- Error 8, Jenkins param renamed, Helm chart breaking change<\/li>
- Error 9, Route 53 needs VPC, private zones are AWS-only<\/li>
- Error 10, Ansible loop cannot parse list, .ini reads everything as string<\/li>
- Error 11, Pi-hole v6 broken, replaced with CoreDNS<\/li>
- Error 12, MetalLB mixed protocol, split TCP and UDP into separate services<\/li>
- Error 13, ConfigMap not found, wrong task order again<\/li>
- Error 14, K3s dir missing on worker, directory not created by agent<\/li>
- Error 15, Registry IP mismatch, MetalLB assigns sequentially<\/li><\/ul>","protected":false},"excerpt":{"rendered":"
Overview This is the most honest article in the series. Every tutorial makes things look easy. You follow the steps, it works,…<\/p>\n","protected":false},"author":2,"featured_media":179,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_rezaops_part":"11","_rezaops_linkedin_url":"https:\/\/www.linkedin.com\/pulse\/part-11-15-real-errors-i-hit-building-home-lab-reza-chegini-t4bqc\/","footnotes":""},"categories":[10,62],"tags":[8,66,110,109],"series":[64],"class_list":["post-177","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","category-home-lab","tag-devops","tag-home-lab","tag-lessons-learned","tag-troubleshooting","series-home-lab"],"jetpack_featured_media_url":"https:\/\/rezachegini.com\/wp-content\/uploads\/2026\/10\/part-11-15-real-errors-cover.png","_links":{"self":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts\/177","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=177"}],"version-history":[{"count":1,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts\/177\/revisions"}],"predecessor-version":[{"id":178,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts\/177\/revisions\/178"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/media\/179"}],"wp:attachment":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=177"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=177"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=177"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fseries&post=177"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}