Skip to main content

NCV GitLab Go-Live Checklists

Production service: code.naudecoreventures.com
Registry: registry.code.naudecoreventures.com
Purpose: A stage-gated acceptance checklist for server deployment, GitLab, Jira, Azure DevOps migration, CI/CD, packages, releases, backups, and endpoint updates.
Every unchecked item is either completed before go-live or recorded as a formally accepted exception with owner, risk, and due date. “We will fix it later” without a Jira item and approver is not acceptance.

Gate A — Inputs, ownership and change control

  • Jira implementation/change Epic exists.
  • Platform owner is named.
  • GitLab administrator is named.
  • Linux administrator is named.
  • Release manager is named.
  • Recovery commander is named.
  • Hetzner Robot administrators and 2FA recovery owners are named.
  • DNS administrators and 2FA recovery owners are named.
  • Jira Cloud administrator is named.
  • Windows runner owner is named.
  • macOS/App Store owner is named.
  • Backup application repository is provisioned.
  • Backup configuration/secrets repository is provisioned separately.
  • SMTP provider/account is approved.
  • Initial GitLab administrator email is approved.
  • Maintenance window and notification channel are defined.
  • Proposed RPO of 24 hours is approved or replaced.
  • Proposed RTO of 8 hours is approved or replaced.
  • Azure DevOps migration scope is signed off.
  • Azure Boards are explicitly retained because Jira is the work-management source of truth, or a separate migration project is approved.
Gate A owner sign-off: ____________________
Date: ____________________

Gate B — DNS and server acquisition

  • The Hetzner auction server is under the correct business account.
  • The approved monthly server cost is recorded and approved.
  • Server serials and both NVMe drive identities are recorded.
  • Reverse DNS is set if required by mail/security policy.
  • code.naudecoreventures.com A record points to the production IPv4.
  • registry.code.naudecoreventures.com A record points to the production IPv4.
  • No conflicting AAAA records exist unless IPv6 is deliberately configured and firewalled.
  • DNS TTL is reduced before cutover and restoration plan records when to increase it.
  • External resolution works from at least two networks.
  • Ports 80 and 443 reach the server for ACME/TLS issuance.
Evidence:
Gate B owner sign-off: ____________________
Date: ____________________

Gate C — Ubuntu, RAID and host security

  • Ubuntu 24.04 LTS installed through Hetzner installimage.
  • Software RAID is enabled.
  • RAID level is 1, never 0.
  • Both NVMe drives are members of every required RAID-1 array.
  • /proc/mdstat shows healthy arrays, normally [UU].
  • Filesystem is ext4 unless an alternative is explicitly approved.
  • Swap and root layout match the implementation runbook.
  • Hostname is set deliberately.
  • Host timezone is UTC and NTP is synchronized.
  • ncvops named administrator exists.
  • At least two emergency administrator public keys are installed.
  • Root SSH login is disabled.
  • Password and keyboard-interactive SSH login are disabled.
  • SSH forwarding is disabled by default.
  • A second SSH session was used to prove access after hardening.
  • UFW/nftables exposes only TCP 22, 80 and 443 publicly.
  • External port scan confirms no internal service is exposed.
  • Fail2ban is active and the sshd jail is enabled/tested.
  • Unattended Ubuntu security updates are enabled.
  • SMART monitoring is active.
  • mdadm monitoring/alerting destination is configured.
  • systemctl --failed returns no unexplained unit failures.
  • Initial server baseline output is attached to Jira.
Evidence:
Gate C owner sign-off: ____________________
Date: ____________________

Gate D — GitLab installation and TLS

  • GitLab EE installed from the official Linux package repository.
  • No paid license is installed unless management deliberately procures one.
  • Exact installed GitLab package version is recorded.
  • external_url is https://code.naudecoreventures.com.
  • registry_external_url is https://registry.code.naudecoreventures.com.
  • GitLab display timezone is Africa/Lusaka.
  • Public sign-up is disabled in configuration and Admin UI.
  • Git over SSH advertises the correct hostname/port.
  • TLS certificate is trusted and has the complete chain.
  • HTTP redirects to HTTPS.
  • Registry /v2/ returns expected 200/401 rather than TLS/5xx failure.
  • SMTP encrypted settings are configured.
  • GitLab test email reaches approved mailbox.
  • Let’s Encrypt auto-renew is enabled or replacement certificate process is documented.
  • GitLab readiness/liveness checks pass.
  • gitlab-rake gitlab:check SANITIZE=true passes.
  • gitlab-rake gitlab:doctor:secrets passes.
  • No internal Prometheus/PostgreSQL/Redis port is public.
Evidence:
Gate D owner sign-off: ____________________
Date: ____________________

Gate E — Identity, groups and permissions

  • Named GitLab administrator accounts created.
  • Emergency root credentials stored in external vault/escrow.
  • 2FA enabled for administrators.
  • 2FA enforced for NCV groups.
  • ncv top-level private group exists.
  • ncv/products exists.
  • ncv/services exists.
  • ncv/platform exists.
  • ncv/archives exists.
  • ncv-releases release-only group exists.
  • Owner membership is minimal.
  • Maintainer membership is restricted to actual technical/release leads.
  • Developers cannot push directly to main.
  • Merge-request approvals are configured according to project risk.
  • v* and other production tags are protected.
  • Release-project main branch is protected.
  • Protected CI variables are available only to protected refs.
  • User deprovisioning process is documented and tested.
  • Token lifetime and review policy is published.
Negative tests:
  • Developer direct push to main is rejected.
  • Developer cannot create protected production tag.
  • Reporter cannot view confidential CI variables.
  • Unauthenticated user cannot browse private source project.
  • Release download policy behaves as designed for a public/anonymous release project.
Gate E owner sign-off: ____________________
Date: ____________________

Gate F — Backup and disaster recovery

  • Restic application repository initialized.
  • Restic configuration/secrets repository initialized separately.
  • Repository passwords/access paths differ.
  • Credentials are stored in mode-0600 root file.
  • Emergency copies are escrowed outside GitLab and the GitLab host.
  • ncv-gitlab-backup script installed.
  • ncv-gitlab-backup-check script installed.
  • Backup systemd service installed.
  • Backup timer installed and enabled.
  • Application backup completes.
  • Configuration/secrets backup completes.
  • Latest snapshots are under 36 hours old.
  • Both repositories pass Restic check.
  • /etc/gitlab/gitlab-secrets.json exists in configuration restore evidence.
  • /etc/gitlab/gitlab.rb exists in configuration restore evidence.
  • SSH host keys are captured.
  • Exact GitLab version/edition is recorded with the backup.
  • Full exact-version restore completed on a fresh isolated host.
  • Repository clone, LFS, Package Registry, Container Registry, Release, channel manifest, login, 2FA/secrets, and pipeline validations passed.
  • Measured RPO/RTO is recorded.
  • Quarterly restore drill is scheduled.
Evidence:
Gate F owner sign-off: ____________________
Date: ____________________

Gate G — Co-located Linux runner

  • GitLab Runner installed from official repository.
  • Runner created with a current glrt- authentication token.
  • Runner scope is group/project-specific, not casually instance-wide.
  • Runner is locked to approved trusted projects.
  • Runner is not available to public/untrusted forks.
  • Docker executor is unprivileged.
  • Docker socket is not mounted.
  • GitLab host data directories are not mounted.
  • Global concurrent = 1.
  • Runner limit = 1.
  • CPU limit is 2.
  • Memory limit is 8 GB.
  • One test job runs.
  • A second job queues rather than running concurrently.
  • GitLab remains responsive during a representative job.
  • Runner migration trigger is documented: move it first if GitLab latency or queues become unacceptable.
Evidence:
Gate G owner sign-off: ____________________
Date: ____________________

Gate H — Jira integration

  • GitLab is externally reachable over trusted HTTPS.
  • GitLab for Jira Cloud app is installed by approved Jira administrator.
  • Intended GitLab groups/projects are connected.
  • Jira issue integration is configured where required.
  • Jira key naming convention is published.
  • Test branch includes a real Jira key.
  • Test commit includes the Jira key.
  • Test merge request includes the Jira key.
  • Pipeline/build/deployment information appears in Jira.
  • Jira remains the authoritative work-management system.
  • GitLab issue creation is disabled by policy where it would split work tracking, or the exception is documented.
Gate H owner sign-off: ____________________
Date: ____________________

Gate I — CI template platform

  • ncv/platform/ci-templates project exists.
  • Common template committed.
  • Java Gradle template committed.
  • Java Maven template committed.
  • Flutter template committed.
  • Deployment template committed.
  • Release template committed.
  • Container BuildKit template committed.
  • Release-tools Dockerfile and helper scripts committed in the CI-template project.
  • release-tools:v1.0.0 image is built, vulnerability-scanned, smoke-tested, and published to the NCV Container Registry.
  • Production release template references the approved release-tools image digest (or a controlled initial tag with a recorded digest-migration action).
  • Template release tag v1.0.0 or approved equivalent is protected.
  • Product projects include immutable template tags, not main.
  • CI lint succeeds for every example.
  • Cache keys are project/ref/tool-version scoped.
  • Test reports and coverage appear in pipeline UI.
  • Artifacts have explicit retention.
  • Deployments use resource_group to prevent overlapping environment changes.
  • Production jobs are manual and protected by branch/tag/variable/access policy.
  • Ordinary merge-request jobs cannot access production/signing secrets.
Gate I owner sign-off: ____________________
Date: ____________________

Gate J — Release project, packages and channels

  • Pilot release project exists under ncv-releases.
  • Source code is absent from the release-only project.
  • Release download visibility is approved and tested.
  • No permanent GitLab credential is required inside customer applications.
  • Generic Package Registry stores the pilot binary.
  • Package filename contains product, version, platform, and architecture.
  • Package SHA-256 sidecar is present.
  • Re-upload with identical SHA is idempotent.
  • Re-upload with different bytes for same version/file is rejected.
  • GitLab Release is created in the release project.
  • Release notes are complete.
  • Release asset link resolves.
  • Experimental manifest points to the immutable package.
  • Beta manifest points to the same artifact URL/SHA.
  • Stable manifest points to the same artifact URL/SHA.
  • No promotion job rebuilds the package.
  • Rollout salt is preserved when increasing percentage for the same artifact.
  • Stable rollback to prior manifest is tested.
  • Release management can see versions and notes in GitLab Releases.
  • Management portfolio view approach is documented: native group/project views initially, optional read-only dashboard later.
Evidence comparison:
Gate J owner sign-off: ____________________
Date: ____________________

Gate K — Windows runner and signing

This gate may follow initial GitLab go-live but must pass before signed Windows customer releases.
  • Separate Windows host/VM provisioned.
  • Supported Visual Studio C++ desktop workload installed.
  • Flutter Windows toolchain passes flutter doctor.
  • GitLab Runner installed as dedicated service account.
  • Runner protected and locked to approved projects.
  • Test build produces expected Flutter Windows output.
  • Installer/package tool is pinned.
  • Code-signing certificate/key custody is approved.
  • Ordinary build job cannot access signing key.
  • Protected signing job can sign.
  • signtool verify /pa succeeds.
  • Timestamping succeeds.
  • Signed package SHA is recorded after signing.
  • Antivirus/EDR test passes.
  • Non-admin install/update test passes.
Gate K owner sign-off: ____________________
Date: ____________________

Gate L — macOS/App Store Connect

This gate is required before Apple deployment.
  • Separate Mac runner or approved hosted Mac exists.
  • Xcode and Flutter versions are pinned/recorded.
  • GitLab Runner runs under dedicated account.
  • Keychain access is limited to protected jobs.
  • App Store Connect API key is protected and externally recoverable.
  • Bundle ID/team/app record match.
  • Signing/provisioning succeeds.
  • TestFlight upload succeeds.
  • Internal tester assignment succeeds.
  • Release notes/metadata flow is documented.
  • App Store production submission remains manual until explicitly approved.
  • Apple credentials are not present on Linux/Windows runners.
Gate L owner sign-off: ____________________
Date: ____________________

Gate M — Azure DevOps pilot inventory

  • Azure organization/project recorded.
  • Repository URL/default branch/size recorded.
  • All branches/tags recorded.
  • Git LFS usage recorded.
  • Submodules recorded.
  • Branch policies recorded.
  • YAML pipelines recorded.
  • Classic pipelines recorded separately.
  • Variable groups inventoried without exposing values.
  • Secure files inventoried.
  • Service connections inventoried.
  • Agent pools/demands inventoried.
  • Environments/approvals/checks inventoried.
  • Azure Artifacts feeds and consumers/producers inventoried.
  • Schedules/webhooks/status checks inventoried.
  • Build/release artifacts retention decision recorded.
  • Azure PR/build history retention method is approved.
  • Rollback owner and freeze window are approved.
Gate M owner sign-off: ____________________
Date: ____________________

Gate N — Azure repository and pipeline pilot

  • Empty GitLab destination project created.
  • Mirror clone completed.
  • All Git LFS objects fetched.
  • Mirror push completed.
  • All LFS objects pushed.
  • Source/destination branch sets match.
  • Source/destination tag sets match.
  • Default branch tip SHA matches.
  • Clean GitLab clone succeeds.
  • LFS checkout and git lfs fsck succeed.
  • Submodules use valid GitLab/approved URLs.
  • Azure YAML pipeline translated to GitLab CI.
  • Classic pipeline behavior explicitly recreated, not silently lost.
  • Variables recreated with correct protected/masked/file type.
  • Service connections replaced with least-privilege credentials.
  • Secure files replaced safely.
  • Test/staging deployment succeeds.
  • Package/release flow succeeds.
  • Azure is frozen for final mirror.
  • Final mirror/verification succeeds.
  • GitLab becomes writable source of truth.
  • Azure repository becomes read-only archive.
  • Rollback window and decision point are recorded.
Gate N owner sign-off: ____________________
Date: ____________________

Gate O — Developer onboarding

  • Developer guide published.
  • Developers can sign in with 2FA.
  • SSH keys added and tested.
  • Git name/email configured.
  • Jira key branch convention understood.
  • Commit/MR title convention understood.
  • Merge-request template active.
  • Developers cannot bypass main controls.
  • Local Java/Flutter toolchain versions match CI policy.
  • Developers know which jobs run on merge requests versus release tags.
  • Developers know not to place secrets in repository/pipeline logs.
  • Developers know releases are immutable and promotion never rebuilds.
  • Support/escalation path is published.
Pilot developer exercise:
  • Create Jira-linked branch.
  • Commit and push.
  • Open MR.
  • Observe tests/reports.
  • Correct a deliberate pipeline failure.
  • Merge after approval.
  • Locate deployment/package/release notes.
Gate O owner sign-off: ____________________
Date: ____________________

Gate P — Desktop updater proof of concept

No endpoint updater is accepted from its marketing/package description alone.
  • Updater candidate decision record exists.
  • Update metadata is signed, not merely hashed.
  • Artifact SHA-256 is verified.
  • Windows platform signature is verified before install.
  • Linux/macOS trust model is documented.
  • Stable/Beta/Experimental channel selection works.
  • Device policy cannot be escalated to Experimental by normal customers unless deliberately allowed.
  • Optional update works.
  • Mandatory update works.
  • Restart now/later works.
  • Background download works.
  • Interrupted network/power test works safely.
  • Corrupt/truncated package is rejected.
  • Wrong SHA is rejected.
  • Invalid metadata signature is rejected.
  • Invalid platform signature is rejected.
  • Locked/running application files are handled safely.
  • Disk-full behavior is safe.
  • Non-admin user behavior is correct.
  • Antivirus/EDR behavior is acceptable.
  • Oldest supported version can upgrade.
  • Rollback/recovery is demonstrated.
  • Multiple app instances are handled.
  • POS active transaction cannot be interrupted.
  • Update success/failure telemetry reaches approved management endpoint.
  • Endpoint contains no permanent personal/project access token.
  • Production approval is signed by platform owner and release owner.
Gate P owner sign-off: ____________________
Date: ____________________

Gate Q — Production cutover

  • All mandatory gates above are signed.
  • Recent application and configuration backups are under 36 hours old.
  • RAID is healthy.
  • Root filesystem is below 70% target and certainly below 80% no-go threshold.
  • GitLab health checks pass.
  • Pilot GitLab pipeline passes.
  • Pilot package/release/channel flow passes.
  • Jira integration passes.
  • Azure final mirror verification passes for cutover wave.
  • Maintenance notification issued.
  • Rollback point/time is recorded.
  • Named incident contact is available during cutover.
  • DNS/TLS/SSH clone from outside network passes.
  • Developer communication confirms GitLab is source of truth.
  • Azure destination for migrated repositories is read-only after validation.
  • Post-cutover monitoring window begins.
Go-live authorization:

Gate R — 24-hour review

  • No unexplained GitLab/registry 5xx spike.
  • No RAID/SMART issue.
  • Storage growth is expected.
  • Backup completed after cutover.
  • Jira development data remains visible.
  • Runner queues are acceptable.
  • Developers report no critical clone/push/MR blockers.
  • Package and release downloads succeed.
  • Azure read-only status remains intact.
  • Any incident has an owner and severity.

Gate S — 7-day review

  • One full weekly security/health review completed.
  • Restic checks pass.
  • Access/Owner/Maintainer review completed.
  • Pipeline duration and queue metrics reviewed.
  • GitLab response time during builds reviewed.
  • Disk/package/artifact growth forecast updated.
  • Azure migration wave lessons incorporated.
  • Developer guide corrections published.
  • No release immutability violation occurred.
  • Scaling trigger not reached, or follow-up server/object-storage ticket exists.

Gate T — 30-day review

  • Monthly GitLab upgrade procedure completed or scheduled.
  • Backup retention verified in both repositories.
  • Restore drill schedule remains active.
  • Token expiry review completed.
  • All pilot migration exceptions closed or re-approved.
  • Remaining Azure migration waves re-baselined.
  • Windows/macOS runner roadmap reviewed.
  • Endpoint updater POC status reviewed.
  • Management release visibility is adequate; dashboard decision recorded.
  • Capacity below scaling thresholds or approved scale task exists.

Immediate no-go list

The production decision is automatically NO-GO when:
  • RAID is not healthy;
  • no offsite application backup exists;
  • no separate configuration/secrets backup exists;
  • no restore drill has passed;
  • public sign-up is enabled;
  • TLS is untrusted/expired;
  • co-located runner can run untrusted projects or privileged Docker;
  • administrator/release-owner 2FA is absent;
  • signing credentials are visible to ordinary pipeline jobs;
  • a package version can be silently replaced with different bytes;
  • Stable promotion rebuilds the release;
  • Azure source/destination refs do not match at cutover;
  • filesystem is above 80%;
  • current backup is older than 36 hours;
  • customer updater POC has not passed security/interruption/restart tests;
  • emergency access depends on one individual.