NCV GitLab Delivery Platform Implementation Plan
For agentic workers: Execute tasks in order. Every task ends with an independently verifiable result. Do not proceed through a failed acceptance gate.Goal: Establish
code.naudecoreventures.com as Naudé Core Ventures’ self-managed GitLab engineering and release platform, migrate Azure DevOps repositories and pipelines, integrate Jira, and implement immutable Experimental/Beta/Stable release channels.
Architecture: One GitLab Linux-package node on the supplied Hetzner RAID-1 server, one constrained co-located Linux Docker runner, external offsite backups, and separate Windows/macOS runners. Source projects publish immutable packages to release-only projects; channel promotion changes signed/verified manifests without rebuilding artifacts.
Tech stack: Ubuntu 24.04 LTS, GitLab EE in Free mode, bundled PostgreSQL/Redis/Gitaly/NGINX, GitLab Runner Docker executor, GitLab Generic Packages, Container Registry, GitLab Releases, Jira Cloud, Bash, JSON/JQ, Java/Gradle/Maven, Flutter, PowerShell, and optional Rust/native desktop updater helper.
Global constraints
- Public GitLab URL is exactly
https://code.naudecoreventures.com. - Container Registry URL is
https://registry.code.naudecoreventures.com. - OS is Ubuntu 24.04 LTS.
- Both NVMe drives use software RAID 1.
- GitLab is the authoritative source/build/release platform; Jira remains the work-management source.
- No plaintext production secret is committed to Git.
- The Linux runner on the GitLab host has concurrency one and is restricted to trusted NCV projects.
- The supplied CPU is treated as a constrained pilot below GitLab’s current single-node baseline in physical-core terms; runner separation is the first scale action.
- Windows native builds run on Windows; Apple builds run on macOS.
- Promotion between release channels never rebuilds or re-signs the artifact.
- Backups leave the physical server and are restore-tested.
- Azure is placed read-only only after repository, LFS, pipeline, package, and deployment verification passes.
Jira delivery model recommendation (plan-only)
Recommended project type: Jira Software, company-managed, using the existingDEV project only after its settings, permissions, and issue-type scheme are verified. Do not create a second project for this program unless the existing project cannot provide the required controls.
Recommended board: One Kanban board filtered to the GitLab delivery component and the label ncv-gitlab-deployment. Use columns Backlog, Ready, In Progress, Validation, Blocked, and Done. Kanban is preferred over Scrum because the program is dependency-led, has infrastructure gates, and does not require sprint commitments. Apply WIP limits to In Progress and Validation; keep blocked work visible rather than hiding it in a separate queue.
Issue-type hierarchy: Jira Cloud’s standard hierarchy is Epic at level 1, standard issues at level 0, and Sub-task at level -1. Therefore:
Workstream issue type for this plan. Represent each workstream as an Epic, use Tasks for executable delivery units, use Sub-tasks for bounded verification or configuration steps, and use Bugs for observed failures linked to the affected Task. Use issue links for dependencies such as blocks and is blocked by; do not encode dependency order only in text.
Required fields on every Task: owner, reviewer, due date, dependency links, acceptance criteria, verification evidence, rollback procedure, affected GitLab group/project, and a single program label. Keep secrets as vault references only; Jira descriptions and comments must not contain secret values.
Proposed Workstream/Epic map
Each numbered implementation task becomes one Jira Task under exactly one Epic. Verification commands, evidence capture, and rollback checks become Sub-tasks only when they can be completed independently; the parent Task remains open until all required evidence is attached and the acceptance statement passes.
Task 1: Establish the implementation record and input inventory
Produces: A Jira implementation epic, owner matrix, access inventory, and rollback contacts.- Create a Jira epic titled
Implement NCV GitLab Delivery Platform. - Create work items for infrastructure, security, backup, Jira integration, Azure inventory, repository migration, pipeline migration, runner setup, release management, endpoint updater POC, and cutover.
- Assign a named owner and reviewer to every work item.
- Record the supplied Hetzner Auction ID
3051857, monthly price, datacenter, IPv4 address, rescue access owner, and support phone password location. - Record DNS administrator, GitLab administrator, Azure DevOps organization administrator, Jira Cloud administrator, SMTP administrator, backup-storage administrator, Windows signing owner, and Apple account owner.
- Create an encrypted secret inventory containing only locations/references, never secret values in Jira.
Task 2: Prepare DNS before installation
Produces: Valid public DNS for GitLab and registry.- Create
Arecordcode.naudecoreventures.compointing to the server IPv4. - Create
Arecordregistry.code.naudecoreventures.compointing to the same IPv4. - Create
AAAArecords only if IPv6 is deliberately configured and firewalled. - Set TTL to 300 seconds during implementation; raise it after go-live.
- Verify authoritative resolution from at least two external resolvers.
- Confirm ports 80 and 443 can reach the server for Let’s Encrypt HTTP validation.
Task 3: Install Ubuntu with RAID 1
Produces: A clean, mirrored Ubuntu 24.04 LTS host.- Activate Hetzner Rescue System and reboot.
- Connect as root using the one-time rescue password.
- Record detected NVMe device names and SMART status.
- Run Hetzner
installimage. - Select Ubuntu 24.04 LTS minimal/base image.
- Set hostname
code.naudecoreventures.com. - Enable software RAID and set RAID level 1 across both drives.
- Use ext4 and leave enough free workspace for GitLab backup generation.
- Complete installation, reboot, and verify the server boots from either mirror member.
- Verify
/proc/mdstat,mdadm --detail, filesystem capacity, and both NVMe devices.
installimage; no production data exists yet.
Task 4: Apply the operating-system security baseline
Produces: Key-only administrative access, firewalls, patching, time synchronization, and hardware monitoring.- Create a named sudo administrator account.
- Install the administrator’s Ed25519 public key.
- Open a second SSH session and verify sudo before changing root/password settings.
- Disable SSH password authentication and direct root login.
- Configure UFW for SSH, HTTP, and HTTPS; do not expose PostgreSQL, Redis, Gitaly, or Registry internal ports.
- Mirror equivalent allow rules in Hetzner Robot Firewall without locking out the active SSH source.
- Install and enable
fail2ban,unattended-upgrades,smartmontools,mdadm,jq,curl,ca-certificates, andchronyor systemd time synchronization. - Configure
mdadmand SMART failure notifications. - Configure Hetzner SysMon for HTTPS and optionally SSH.
- Keep the OS clock in UTC; configure GitLab’s display timezone as
Africa/Lusaka.
Task 5: Install and configure GitLab
Files used:config/gitlab.rb.example, config/sshd-hardening.conf, scripts/gitlab-health-check.sh.
Produces: A healthy GitLab instance at the required URL.
- Install GitLab EE from the official Linux package repository without activating a paid license.
- Set
EXTERNAL_URL=https://code.naudecoreventures.com. - Apply the supplied
gitlab.rbbaseline. - Enable automatic Let’s Encrypt certificates for GitLab and the registry.
- Disable public sign-up and require administrator-created accounts.
- Configure outbound SMTP and verify invitation/password-reset delivery.
- Enable the Container Registry at
registry.code.naudecoreventures.com. - Run
gitlab-ctl reconfigureandgitlab-ctl status. - Run
gitlab-rake gitlab:check SANITIZE=true. - Change the initial root password and place it in NCV’s approved password vault.
- Create a second administrator account and verify recovery access.
docker login registry.code.naudecoreventures.com reaches the registry authentication flow.
Rollback: Restore /etc/gitlab configuration backup or reinstall the same GitLab version before production data is added.
Task 6: Configure identity, permissions, and project structure
Produces: Standardized NCV groups and least-privilege access.- Enforce 2FA for all human users.
- Disable user-created top-level groups and unrestricted project creation.
- Create private top-level group
ncv. - Create subgroups
ncv/platform,ncv/infrastructure, andncv/products. - Create a release-only top-level group
ncv-releases; make only its intended release projects public. - Create
ncv/platform/ci-templatesand tag the first template versionv1.0.0. - Create
ncv/infrastructure/gitlab-platformand commit this handbook after removing any secrets. - Define roles: Owner for a very small administrative set, Maintainer for technical leads, Developer for engineers, Reporter for read-only management/audit users.
- Protect
main,release/*,v*, andcandidate-*refs. - Prevent direct pushes to
main; merge through merge requests only.
main, create a stable tag, or alter group settings.
Rollback: Group owners can correct memberships and branch/tag rules; no code migration is required yet.
Task 7: Implement offsite backup before migration
Files used:scripts/backup-gitlab.sh, config/gitlab-backup.service, config/gitlab-backup.timer.
Produces: Encrypted offsite backups and a proven restore process.
- Provision two encrypted offsite Restic repositories: one for application backups and one, with separate access/password, for GitLab configuration and decryption secrets.
- Store both repository locations and separate credentials in
/root/.config/ncv-gitlab-backup.envwith mode0600; escrow recovery credentials outside the GitLab host. - Install Restic.
- Install the supplied backup script and systemd service/timer.
- Run one manual GitLab application backup.
- Run
gitlab-ctl backup-etcand include/etc/gitlab/config_backup, TLS material, and SSH host keys in encrypted offsite backup. - Verify both remote Restic repositories and perform
ncv-gitlab-backup-check --check. - Build a temporary restore host or VM with the exact same GitLab version and edition.
- Restore application backup and GitLab secrets/configuration.
- Clone a test repository, inspect a release/package, and run a login test.
- Record restore evidence in Jira.
Task 8: Install the constrained Linux runner
Files used:config/gitlab-runner-config.toml.example.
Produces: One trusted Docker runner for Java, Flutter checks, Linux builds, and deployment scripts.
- Install Docker Engine and GitLab Runner.
- Register an instance/group runner tagged
linux-smallandtrusted. - Use the Docker executor with unprivileged containers.
- Set global
concurrent = 1and runnerlimit = 1. - Limit each job to two CPUs and 8 GB memory.
- Enable per-build Docker networks.
- Persist only the runner cache directory.
- Do not mount
/var/run/docker.sockinto general jobs. - Do not allow forks or untrusted public projects to use this runner.
- Run a CPU/memory test job while observing GitLab response time.
Task 9: Connect Jira Cloud
Produces: Jira development-panel visibility for GitLab activity.- Confirm GitLab is public over HTTPS with a complete trusted certificate chain.
- Install/configure the GitLab for Jira Cloud app using an approved Jira administrator.
- Connect the intended GitLab groups/projects.
- Configure project-level Jira issue integration where required.
- Create a test branch containing a real Jira key.
- Commit, push, open an MR, run a pipeline, and create a test deployment.
- Confirm Jira shows branch, commit, MR, pipeline, and deployment data.
- Publish the mandatory naming convention to developers.
Task 10: Create CI templates and the release model
Files used:ci/templates/*, scripts/publish-generic-package.sh, scripts/promote-channel.sh, templates/channel-manifest.schema.json, and templates/release-notes.md.
Produces: Reusable CI and an immutable release lifecycle.
- Commit the templates, helper scripts, and
ci/images/release-tools/Dockerfiletoncv/platform/ci-templates. - Build, scan, verify, and publish
release-tools:v1.0.0to the NCV Container Registry; record and later pin its digest. - Protect template tags; projects include templates by immutable tag, not
main. - Create one
ncv-releases/<pilot-product>project containing only release material. - Configure its Package Registry for the intended anonymous/authenticated download policy.
- Protect the release project default
mainbranch; store manifests beneathchannels/. - Configure cross-project publish credentials using a job token allowlist or narrowly scoped project access token.
- Publish a test immutable package.
- Create a GitLab Release with release notes and package asset links.
- Publish Experimental manifest pointing to the package.
- Promote the exact SHA to Beta, then Stable, without rebuilding.
- Verify each manifest, URL, package checksum, and GitLab release entry.
Task 11: Inventory Azure DevOps
Produces: Complete migration inventory and wave plan.- List organizations, projects, repositories, default branches, branch policies, tags, Git LFS usage, submodules, and repository sizes.
- List YAML and classic pipelines, schedules, triggers, environments, approvals, agent pools, artifacts, caches, variable groups, secure files, and service connections.
- List Azure Artifacts feeds and package consumers/producers.
- List webhooks, status checks, deployment scripts, and external integrations.
- Classify repositories into pilot, standard, high-risk, archived, and retained-read-only.
- Record every production deployment pipeline and its rollback method.
- Choose migration waves and freeze owners.
Task 12: Migrate the pilot repository
Files used:scripts/migrate-azure-repo.sh, scripts/verify-repo-migration.sh.
Produces: A GitLab repository with all Git refs and LFS objects.
- Create empty destination project under
ncv/products. - Mirror-clone the Azure repository.
- Fetch all Git LFS objects.
- Push all refs to GitLab.
- Push all LFS objects to GitLab.
- Compare branch and tag sets.
- Compare the default branch tip SHA.
- Clone the GitLab repository into a clean directory and build locally.
- Keep Azure PR/build history read-only; do not pretend Git mirror migration moved it.
Task 13: Convert and validate the pilot pipeline
Produces: GitLab CI parity with the Azure pipeline.- Map Azure stages/jobs/steps to GitLab stages/jobs/scripts.
- Map Azure agent pools to GitLab runner tags.
- Recreate variables in GitLab group/project CI/CD settings; never copy secrets into YAML.
- Recreate secure files in GitLab Secure Files or approved vault delivery.
- Replace Azure service connections with narrowly scoped GitLab/deployment credentials.
- Recreate schedules and manual inputs.
- Produce JUnit XML and coverage artifacts where supported.
- Build and deploy to a non-production test environment.
- Compare outputs with Azure by file name, size, version metadata, and checksum where deterministic.
- Run smoke and integration tests.
- Run the release pipeline through Experimental and Beta; do not deploy Stable yet.
Task 14: Pilot cutover
Produces: One product operating fully from GitLab.- Announce a repository freeze window in Jira and team communication.
- Disable merges in Azure for the pilot repository.
- Fetch and push the final delta to GitLab.
- Run migration verification again.
- Change developer remotes to GitLab.
- Enable GitLab as the only production pipeline.
- Run a tagged release and promote the exact artifact to Stable.
- Confirm package download, release notes, deployment, monitoring, and rollback.
- Mark Azure repository and pipeline read-only; retain history for the agreed retention period.
- Record cutover evidence and owner sign-off in Jira.
Task 15: Migrate remaining repositories in waves
Produces: Azure DevOps retirement without a big-bang migration.- Repeat Tasks 12–14 by migration wave.
- Do not mix unrelated high-risk production systems in one wave.
- Validate backup growth and runner load after every wave.
- Add a separate Linux runner before increasing concurrency on the GitLab host.
- Archive rather than migrate dead repositories unless legal/business retention requires active hosting.
- Revoke Azure service connections and tokens only after dependent systems are verified.
Task 16: Implement Windows and Apple runners
Produces: Native cross-platform builds.- Install a dedicated Windows runner tagged
windows-x64. - Install Flutter, Visual Studio Build Tools/workloads, installer tooling, and code-signing access.
- Run the runner under a restricted service account; protect the runner directory.
- Store signing credentials only in protected variables/secure files and expose them only on protected refs.
- Build, sign, package, and verify one Windows release.
- Install a dedicated macOS runner tagged
macos-arm64when Apple delivery is required. - Configure Xcode, Flutter, Fastlane, keychain access, App Store Connect API key, TestFlight groups, and signing.
- Verify macOS runner user-session and keychain behavior after reboot.
Task 17: Endpoint updater proof of concept
Produces: A go/no-go decision for automatic customer updates.- Implement the updater behind an application-owned interface so the package can be replaced later.
- Validate channel selection from server policy rather than allowing ordinary customers to select Experimental.
- Check current version and fetch release notes.
- Download to a staging path and verify file length, SHA-256, and publisher signature.
- Test optional update, mandatory update, restart-now, restart-later, silent idle update, and failed update recovery.
- Test power/network interruption during download and install.
- Test an application process holding files open.
- Test downgrade/rollback policy.
- Test standard-user installation permissions.
- Test antivirus/EDR and Windows SmartScreen behavior with signed binaries.
- Test Linux executable permissions and publisher authenticity.
- Capture update success/failure telemetry without collecting customer secrets.
Task 18: Operational handover and go-live
Produces: A supportable platform rather than an undocumented installation.- Run all checks in
docs/09-go-live-checklists.md. - Confirm backup and restore evidence is current.
- Confirm administrator break-glass access.
- Confirm monitoring and alert routing.
- Confirm upgrade owner and maintenance policy.
- Train developers using
docs/04-developer-guide.md. - Train release managers using
docs/07-release-package-update-guide.md. - Train infrastructure staff using the operations and DR runbooks.
- Store this handbook in
ncv/infrastructure/gitlab-platformand tag the approved baseline. - Close the Jira implementation epic only after owner and reviewer acceptance.