DevNJ Make your business ease
Column /

9 ways automated website deployments break, and how to prevent them: deletions, overwrites, rollback and post-release checks (2026)

Column › Site operations

9 ways automated website deployments break, and how to prevent them: deletions, overwrites, rollback and post-release checks (2026)

Automating how you publish a website removes many of the slips people make when they do it by hand. In exchange, when the automation itself is wrong, it repeats the same mistake reliably, every time. Transfer an empty source folder and production files disappear. Let a failed build carry on and a broken page goes live.

This article sorts the ways the deployment mechanism itself breaks into nine types and sets out how to prevent each one. Checking the content of articles is covered in "Automated blog publishing needs a quality gate", updates, backups and uptime monitoring in "Website maintenance automation", and the overall picture in "Automating website operations starts with an inventory". Here the focus is the step that delivers files to production and switches them on.

Main sources for this article: the official rsync manual (--delete, --dry-run, --max-delete and exit values), the GNU Bash manual, GitHub Docs (Actions concurrency, deployment environments, billing), web.dev on HTTP caching, Google Search Central on pausing a site and controlling indexing, DORA's software delivery metrics, Google's SRE Workbook, an order from the US Securities and Exchange Commission (SEC) and GitLab's incident report, all checked on 31 October 2026.

What this article covers Key points
Where things break Nine types across build, transfer, switch and after release (sections 2–3)
How not to delete files Dry run, exclusions, a deletion limit, a source check (section 4)
How to stop on failure Stop on exit codes; never publish a half-finished state (section 5)
How to stay reversible A directory per release and a link switch (section 10)
What it costs Three levels, 6 to 80 hours to set up, €0 to €80 a month (section 12)
Where to start A four-week plan and a checklist (sections 16 and 18)

1. What is changing: automation moves where things break

More businesses are automating how they publish. Push a change to GitHub and the build, tests and transfer run on their own; that is no longer just for large companies. GitHub's pricing includes $12 of compute a month even on the Free plan. At $0.006 a minute on a standard Linux runner, that is roughly 2,000 minutes. Standard runners are free for public repositories.

Failures move from people's hands into the pipeline

With manual publishing, a failure usually happens once. Forget to upload a file, notice later, fix it, done. With automation, the same configuration error repeats on every release. And because no one is watching the screen, it is noticed later.

DORA measures how often releases fail and how fast you recover

DORA (DevOps Research and Assessment) at Google Cloud recommends five metrics for software delivery: how long a change takes to reach production, how often you deploy, how long it takes to recover from a failed deployment, the share of deployments that need immediate intervention (change fail rate), and the share of deployments that are unplanned fixes. DORA warns that turning these into blanket targets invites gaming. For a small site, making it quick to recover comes before deploying more often.

PHP 8.2's security support also ends on 31 December 2026. The months when you upgrade versions are the months when deployment pipelines break most easily.

A deployment runs through four stages: build, transfer, switch and after release. Build is prone to dependency mismatches and failed builds; transfer to unexpected deletions and overwritten config; switch to database migrations and concurrent runs; after release to caching, missed checks and no way back. 1. Build 2. Transfer 3. Switch 4. After Versions Failed build Deletions Overwrites DB changes Overlaps Caching No checks No way back Top: stages. Below: what tends to break there
Figure 1: The four stages of a deployment and nine ways it breaks (DevNJ's classification, as of 31 October 2026)

As the figure shows, most failure types cluster around the transfer and the switch. Put checks and stop conditions there and six of the nine can be caught early.

2. What deployment covers, and where other checks take over

In this article, deployment means taking changes from your machine or repository, delivering them to the production server and switching them on. It includes the build (generating CSS and JavaScript, installing dependencies), the transfer, the switch and checks after release.

How it differs from other processes

Pre-release checks come in two kinds: those that look at the content, and those that look at how it is delivered. This article covers only the second.

Process What it checks Covered in
Content quality gate Length, sources, wording, images Publishing quality gate
Deployment safety Deletion, overwrites, stopping, switching, rollback This article
Maintenance Updates, backups, uptime, certificates Maintenance automation
Speed monitoring Core Web Vitals over time Core Web Vitals monitoring

Drawing the lines clearly avoids running the same check in two places and doing both badly.

Manual or automated, the failure types are the same

Whether someone types commands from a runbook or a push to GitHub triggers everything, the ways it breaks are the same. What differs is who notices, and when. The more automated you are, the more people need to decide in advance when to stop and whom to alert.

3. A map of the nine failure types

Here are the nine types in one table: what happens, when you notice and what prevents it.

Type What happens When you notice Main safeguard
1. Unexpected deletion Production-only files vanish A user reports it Dry run, exclusions, limit
2. Overwritten config or secrets Production gets local settings Errors right after release Keep config out of the transfer
3. Publishing a failed build Old CSS, missing files Layout complaints Stop on exit codes
4. Interrupted transfer Old and new files mixed Errors on some pages Switch from a separate directory
5. Caching Old pages linger, versions mix Differs per visitor Fingerprinted names, revalidate HTML
6. Database migration Old code no longer runs When you try to roll back Backup first, staged changes
7. Version mismatch Works locally, fails live 500 errors after release Lockfiles, resolve for production
8. Colliding automation Two releases run at once Intermittent faults Concurrency control, locks
9. No rollback, no checks No way back, nobody notices Hours or days later Keep releases, smoke tests

Types 1 to 4 come from carrying on without noticing a failure, types 5 to 7 from gaps between local and production assumptions, and types 8 and 9 from operating rules nobody agreed on.

4. Types 1 and 2: unexpected deletions and overwritten config or secrets

These two do the most damage and are the easiest to prevent. Both happen when the line between what is transferred and what is excluded is vague.

What rsync --delete actually deletes

rsync is a common tool for transferring files to a server. With --delete, it removes files from the destination that do not exist in the source. The official manual says plainly that the option "can be dangerous if used incorrectly" and recommends a --dry-run (-n) first to see which files will be deleted. More files live only in production than people expect, and anything you forget to exclude is gone on the next release.

Production-only files Examples How to protect them
Config and secrets .env, wp-config.php Exclude; never transfer
User uploads uploads/, storage/app/public Keep outside the deploy dir, link in
Data the server creates Fetched rates or weather, generated images Exclude; separate job and location
Logs and temp files storage/logs/ Exclude
Saved backups .env.bak-date Exclude; keep outside the web root

According to the manual, excluded files are also protected from deletion (unless you add --delete-excluded). Your exclusion list doubles as your "do not delete" list.

What happens when the source is empty

The worst case is a source folder that is empty or points somewhere else. Send an empty folder with --delete and the destination is made empty to match.

In January 2015, users reported on GitHub that a line in the Linux version of the Steam game platform, rm -rf "$STEAMROOT/"*, became a command to delete everything from the root directory when the variable was empty, and their files disappeared.

The fix is simple. Before transferring, check that a file that must exist in the source (such as index.php) is there, and stop if it is not. Writing path variables as ${DIR:?} also makes the script stop when they are empty.

Put a ceiling on deletions

rsync has a --max-delete=NUM option. According to the manual, once deletions exceed that number, all further deletions are skipped and rsync exits with code 25. With --max-delete=0, it deletes nothing and just warns about extraneous files.

A normal release removes a handful to a few dozen files. Set the limit a little above your usual number and mass deletions are stopped before they happen.

Do not overwrite secrets, and do not leak them

Transfer a config file and your local development values overwrite production. Debug output can switch on and show configuration values on error pages. Laravel's official documentation says APP_DEBUG must always be false in production, and warns that moving index.php to the project root exposes sensitive configuration files. Keep the application outside the web root and expose only public/.

GitHub's push protection blocks pushes that contain secrets. Production keys used in CI can be stored as environment secrets, which only jobs that have passed protection rules such as approval can read. OWASP's cheat sheet likewise builds on keeping secrets separate from code and giving them the narrowest possible access.

What happened in DevNJ's own operations

On DevNJ's own site there were moments when uncommitted changes nearly went live, when a locally generated cache listing development-only packages caused an error in production, and when data the server generates every hour was nearly wiped by --delete. Releases are now exported from a commit with git archive, and the cache and server-generated data are excluded. Most of these were caught by reading the dry run's list of planned deletions.

5. Types 3 and 4: publishing a failed build and interrupted transfers

The next most common type is a failure partway through, after which the process carries on to the end. By default, a shell script moves to the next line even when a command fails.

Stop on exit codes

Every command returns an exit code: 0 for success, anything else for failure. According to the GNU Bash manual, set -e stops the script when a command exits with a non-zero status, set -u treats unset variables as errors, and set -o pipefail makes a pipeline fail if any command in it fails.

Setting What it stops Without it
set -e Anything after a non-zero exit Transfer runs after a failed build
set -u Use of unset variables A mistyped variable becomes empty
set -o pipefail Failures inside a pipe build | tee log counts as success
Source check A missing required file Production is synced to an empty folder

set -e has exceptions. It does not stop on a failing command inside an if condition or partway through a list joined by && or ||. For important steps, write an explicit exit 1 on failure.

Read rsync's exit codes

rsync returns different exit codes for different failures. These are the ones that come up most in deployments, taken from the manual's list.

Exit code Meaning What to do in a deployment
0 Success Go to the next step
23 Partial transfer due to error Stop and find which files
24 Source files vanished mid-transfer Suspect a running build; stop
25 Deletion limit reached A person reviews the list
30, 35 Timeouts Check the partial state before retrying

The key point is never to treat "partly transferred" as success. Codes 23 and 24 mean production may now hold a mix of old and new files.

With zero planned deletions, continue. From one deletion up to the limit, review the list before continuing. Above the limit, stop. An exit code of 0 means go to the next step; any other code means stop and have a person check. Planned deletions Go Review list Stop 1 file Limit Exit code 0: go Not 0: stop, check
Figure 2: When to stop around the transfer (source: --max-delete and exit values in the rsync manual, checked on 31 October 2026; the limit itself is DevNJ's approach)

The top row is the rule for the number of deletions counted from the dry run. The bottom row is the rule for each step's exit code. Erring on the side of "stop" produces the occasional false alarm, but not accidents.

An interrupted transfer mixes old and new

If you transfer straight into the production directory, an interruption leaves a mix of old and new files. rsync's --delay-updates puts all updated files into place at the end, which in the manual's words only makes the update "a little closer to atomic". The reliable approach is the one in section 10: put the release in a separate directory and switch to it in one step.

6. Type 5: caches keep old pages alive

A release can succeed and visitors still see the old site. The usual cause is caching. Caches live in three places, the browser, the CDN and the application, and each is cleared and behaves differently.

Browser and CDN caches

web.dev's guide to the HTTP cache recommends Cache-Control: max-age=31536000 (one year) for URLs whose content never changes, and changing the URL when the content changes. If the file name includes a string derived from its content (a fingerprint), as in app.3f9a1c.css, each new version gets a new URL and never collides with the old cache.

Resources whose URL cannot change, like HTML, get no-cache, which makes the browser check with the server before every use. Combined with an ETag, an unchanged page costs only a small 304 response.

Resource Recommended setting Why
Fingerprinted CSS, JS, images max-age=31536000 New content means a new URL
HTML no-cache plus ETag Checked each time; 304 if unchanged
Images and PDFs without fingerprints A short max-age The name stays the same when replaced
HTML behind a CDN Purge on release The CDN keeps serving the old version

If a cached old HTML page requests a CSS file under an old name that the build has deleted, the layout breaks. Keeping the previous release's assets for a few days avoids this.

Application caches

Laravel's official documentation recommends running php artisan optimize during deployment to cache configuration, routes and views. But once configuration is cached, .env is no longer loaded, and calls to env() outside config files return null. It is a classic way for code that works locally to lose its values in production.

Queue workers (background jobs) also keep running the old code, so Laravel advises restarting them after a release with php artisan reload.

7. Type 6: database migrations

A migration is a change to the database structure: adding a column, renaming it, changing its type. You can roll files back to the previous release, but the database structure does not roll back on its own.

Spot the changes you cannot undo

Dropping or renaming columns and tables, or changing types, leaves the old code looking for a column that is no longer there. Roll back only the files and the old code now runs on the new structure and fails.

Change Does the old code still run? Safe approach
Add a nullable column Yes Release as normal
Add a table Yes Release as normal
Rename a column No Four steps: add, write both, migrate, drop
Drop a column No Drop after releasing code that no longer uses it
Change a type It depends Add a new column and migrate

This "add, migrate, then drop" approach is what Martin Fowler describes as Parallel Change. By never changing structure and code in the same release, you can always roll back one version.

Use the safeguards for running in production

Laravel's migrations ask for confirmation before running in production. Automated releases skip that with --force, so always put a backup step before it. When several servers deploy at once, --isolated takes a lock so that only one runs the migration.

On 31 January 2017, an engineer at GitLab.com accidentally deleted about 300 GB of production database data. According to the incident report, about six hours of data was lost (around 5,000 projects, 5,000 comments and 700 users). The scheduled backups had been failing because of a tool version mismatch, and the failure emails had been rejected by the receiving server. Backups are only useful once you have checked both that they run and that failures reach someone.

8. Type 7: mismatched dependency and PHP versions

It works locally and fails in production. Most often, local and production run different versions of PHP or of the dependencies.

Pin versions with lockfiles

npm's npm ci will not run without a lockfile (package-lock.json), exits with an error if it does not match package.json, and never writes to the lockfile. It also removes any existing node_modules first. Use npm ci rather than npm install for release builds and you get the same versions as on your machine.

Composer pins PHP dependencies in composer.lock. Its platform setting lets you resolve dependencies for the production PHP version even when your local PHP is newer. The official documentation notes that this can hide real incompatibilities and recommends also running check-platform-reqs during deployment.

PHP support periods

According to the PHP website, security support for each branch ends as follows.

Branch Released Active support until Security support until
8.2 December 2022 31 December 2024 31 December 2026
8.3 November 2023 31 December 2025 31 December 2027
8.4 November 2024 31 December 2026 31 December 2028
8.5 November 2025 31 December 2027 31 December 2029

Sites on PHP 8.2 need an upgrade plan within the next two months. Align the version in four places, local, CI, staging and production, before you move. Upgrade one place first and you get exactly this type of fault.

On shared hosting, the command-line PHP and the web PHP are often different versions. When only cron jobs fail, check this first.

9. Type 8: automated jobs that collide

As automation grows, publishing is no longer a single process. Automatically generated articles, scheduled data fetches and manual fixes all try to write to the same repository or server at the same time.

Control concurrency in CI

GitHub Actions has concurrency control. According to GitHub Docs, runs in the same group have at most one running and one pending at a time; a new run cancels the existing pending one. With cancel-in-progress: true, the running one is cancelled too.

For production releases, it is safer not to cancel a release that is already running, because stopping halfway causes type 4, mixed old and new files. To keep waiting runs, queue: max lets up to 100 wait in line, but it cannot be combined with cancel-in-progress.

Setting Behaviour Good for
None Any number run at once Not suitable for production releases
concurrency only One running, one waiting Releases where only the latest matters
cancel-in-progress: true Cancels the running one too Previews and tests
queue: max Up to 100 processed in order Releasing every change in turn

Colliding writes to the repository

When an automated article job and a person both push to the same branch, the second push is rejected. As the Git documentation explains, a push to a branch whose history has moved on is refused as it stands. Forcing it overwrites the other side's changes.

Rebasing onto the latest version before pushing, and releasing from there, prevents this. DevNJ's own site has a rule to rebase before pushing so that automated column generation and manual fixes do not collide.

Cron jobs on the server can also rewrite files mid-release. Keep the places scheduled jobs write to outside the area a release replaces.

10. Type 9: no way back, and no checks after release

The last type matters after something breaks: there is no way to roll back, or nobody notices it broke.

A directory per release and a link switch

The basis of an easily reversible setup is to put each version in its own directory and switch only the link the site is served from. Put the new version in releases/20261031-1015/, and once it passes checks, point the current link at it. To roll back, point the link at the previous release.

The Linux manual states that rename replaces the target atomically, even if it already exists. Create the new link under another name and swap it in with rename, and there is no in-between state. The PHP deployment tool Deployer works this way and keeps the last 10 releases by default.

Switching method How long versions mix How to roll back Suited to
Overwrite production directly The whole transfer Re-transfer the old version Small static sites
Stage, then replace in one go Briefly Re-transfer the old version Small sites on shared hosting
Releases plus link switch Almost never Switch the link back PHP and CMS sites
Two environments swapped Never Switch traffic back Sites that cannot go down

The last method is known as blue-green deployment. Whatever the method, database schema changes are not undone by switching back; combine it with the approach in section 7.

As DevNJ estimates, restoring from a backup takes up to 240 minutes, fixing by hand and re-transferring up to 120 minutes, redeploying the old version up to 30 minutes, and switching the link back to the previous release under 1 minute. Backup restore ≤240 min Hand fix ≤120 min Redeploy old ≤30 min Switch link <1 min 0 min
Figure 3: Estimated time to recover by rollback method (DevNJ's working estimates; varies with site size and connection, as of 31 October 2026)

Time to roll back is time your visitors spend looking at errors. A setup you can roll back with a link switch also shortens the time spent hesitating, because "roll back first, investigate later" becomes an option.

When rolling back made things worse

According to a 2013 SEC order, the US firm Knight Capital deployed new code to eight servers in July 2012, but one server was missed and no second technician reviewed the deployment. On 1 August, old functionality on that server sent more than 4 million orders in 45 minutes, and when staff removed the new code from the seven correct servers during the response, the damage spread. Losses exceeded $460 million. Having no reconciliation of what was deployed and no agreed way back is just as risky for a website.

Smoke tests straight after release

Straight after a release, automatically check the response code of key pages and a fixed text string on them. This is called a smoke test. Laravel includes a health route at /up that returns 200 if the application boots and 500 if it throws an exception.

What to check How If it fails
5 to 10 key pages Do they return 200? Switch the link back
Text on the page Is the company name or heading there? Suspect a broken layout
Health route Does /up return 200? Check config and caches
Test form submission Does the email arrive? Check mail settings
noindex Is there none in production? Remove it and release again
robots.txt 200 with the right content? Fix before crawling stops

The last two rows affect search directly. Google's guidance on pausing a site warns that serving robots.txt with a 503 blocks all crawling. A noindex carried over from staging makes pages drop out of search results.

Straight after release, automatically check key pages' responses and text. Within 5 minutes, read the error log. Within 30 minutes, send a test form. The next day, check Search Console for errors and 404s. After 7 days, clean up old releases. At once Key pages and text Automatic; roll back on failure 5 min Read the error log 500s and exceptions 30 min Send a test form Does the email arrive? Next day Check Search Console New errors and 404s 7 days Clean up old releases Keep the last few
Figure 4: A schedule for post-release checks (DevNJ's working practice, informed by Google Search Central guidance on pausing a site, checked on 31 October 2026)

It is fine to automate the immediate check all the way to rolling back without waiting for a person when it fails, because that only touches files and the link, never the database.

If you take the site down for a release, Google's guidance is to return 503 for no more than about 1 to 2 days. Do not leave a maintenance page up over a weekend.

11. Staging and canaries: how much does a small site need?

A copy of production for checking is called staging; releasing a new version to a small share first and watching it is called a canary. Even small sites have cases where each is needed and cases where it is not.

Protecting staging

Use password protection rather than noindex to keep staging out of search results. If you do hide it with noindex, check in every section 10 smoke test that it has not been carried over to production.

Canaries mean releasing to a few first

Google's SRE Workbook defines canarying as "a partial and time-limited deployment of a change in a service and its evaluation", and its worked example uses a 5% canary population. It also recommends running only one canary at a time.

On 19 July 2024, a security product update was pushed to everyone at once, and Microsoft estimated that about 8.5 million Windows devices could not start. If you update several sites with the same pipeline, release to one site first and check it before rolling out to the rest.

Type of site Staging Release to some first Why
Static company site Previews will do Not needed Easy to roll back
CMS site Recommended Not needed Check plugins and PHP
Enquiries or bookings Essential Not needed A broken form loses business
Online shop Essential Recommended Check payments and stock
Bulk updates across sites Essential Essential Stop after the first one

The premise is that staging runs the same PHP version and the same kind of configuration as production. Staging on a different version misses the mismatches in section 8.

12. Costs and effort

The cost of safer deployments depends a great deal on how far you go. The table below is DevNJ's estimate for one PHP or CMS site.

Level What it includes Set-up effort Monthly (€)
1. Safer scripts set -euo pipefail, source check, dry run, exclusions, deletion limit, smoke test 6–12 hours 0
2. CI and release switching Automated builds, lockfiles, concurrency control, release directories, alerts 16–32 hours 0–20
3. Staging and database Staging, migration procedure, pre-release backups, automatic rollback 40–80 hours 20–80

For a small site, GitHub Actions usage usually stays within the free allowance; most of the monthly cost is a staging server and monitoring.

Outsourced at €80 an hour, set-up costs come out as follows. Yen figures are approximate, at ¥175 to the euro.

Level Set-up (€) Set-up (approx. ¥) First-year total (€)
1 480–960 ¥84,000–¥168,000 480–960
2 1,280–2,560 ¥224,000–¥448,000 1,280–2,800
3 3,200–6,400 ¥560,000–¥1,120,000 3,440–7,360

The first-year total is set-up plus 12 months of running costs. Level 1 costs almost nothing and prevents most accidents, so it is worth putting on every site first.

Weigh it against what a breakage costs. A site that gets 20 enquiries a month, each worth €500, loses about €1,000 of opportunities if its form is down for three days. If uploaded images are deleted and must be recreated, add that work too.

13. Common mistakes

The Steam, GitLab.com and Knight Capital incidents in this article correspond to types 1, 6 and 9. Here are the mistakes most often seen on small sites.

Mistake What happens Prevention
A missing exclusion Uploaded images disappear Fix the exclusion list in the runbook
Not watching build failures The site goes live with old CSS Stop on exit codes
Calling env() directly Values are null after config caching Read through config files
noindex from staging Production drops out of search Check automatically after release
Uncommitted local changes Unintended changes go live Export from a commit

Every one of these happens when the "does it stop?" and "can we roll back?" checks are skipped after the mechanism is built.

14. Japan and Europe compared

Running sites in both Japan and Europe changes some deployment assumptions. These are differences in conditions, not in quality.

Item Common in Japan Common in Europe (Netherlands)
Server Shared rental hosting Managed VPS or cloud
Work on the server SSH, but limited tools Deployment tools easy to install
Main guidance IPA's 20 points for site operation NCSC principles, ENISA's 12 steps
Breach reporting Preliminary report to the PPC in 3–5 days Supervisory authority within 72 hours
Release timing Japanese business hours Allow for a 7–8 hour time difference

On shared hosting, tools such as Composer may not exist on the server. In that case, build dependencies for the production version locally or in CI and send them together.

The Netherlands is 8 hours behind Japan in winter and 7 in summer. A 10:00 release in the Netherlands is 18:00 or 17:00 in Japan, so agree in advance which side handles post-release checks. If leaked secrets involve personal data, Article 33 of the GDPR may require notifying the authority within 72 hours.

15. Three cases (fictional companies)

Three fictional companies show how the same principles lead to different answers.

Company A: a Japanese manufacturer's English site for Europe (static)

A static site updated about twice a month; level 1 is enough. rsync gets exclusions, a dry run and a deletion limit, and the 10 key pages are checked after release. To roll back, the previous commit is rebuilt and sent, within about 30 minutes.

Company B: a Japanese food webshop in the Netherlands (CMS)

With 30 to 50 orders a day, it goes to level 3: check on staging, back up the database just before release. Releases happen only on Tuesday or Wednesday mornings.

Company C: a bilingual Japanese–Dutch professional services firm (Laravel)

Automated articles and manual fixes share one repository, so at level 2 concurrency control and rebasing before pushing became part of the procedure. If the smoke test fails, the link switches back automatically, and the Japan side checks Search Console the next morning.

All three started with the checks and stop conditions of level 1, then added what their kind of site needed.

16. Implementation: a four-week plan

Safer deployments can be put in place in about four weeks, with a deliverable for each week.

  1. Week 1: write down the current release procedure. Which commands send what, and where, on one page
  2. Week 1: list the files that exist only in production. Config, uploads, server-generated data, logs (the table in section 4)
  3. Week 2: add set -euo pipefail and a source check to the script
  4. Week 2: add a step to review planned deletions from the dry run, and a deletion limit
  5. Week 3: move to release directories and a link switch
  6. Week 3: add a smoke test that rolls back on failure
  7. Week 4: set up concurrency control and rebase-before-push
  8. Week 4: break it on purpose. Break the build, empty the source, roll back

The last step matters most. Whether the safeguards really stop things is something you only learn by making them fail.

17. What not to do

Settings and decisions to avoid when automating deployments:

Don't Why
Use --delete without a dry run Production is synced without seeing what goes
Include .env in the transfer Local values overwrite production settings
Let the script continue after failures Broken states go live
Cancel a running production release Old and new files end up mixed
Change structure and code in one release Rolling back files no longer works
Resolve conflicts with a force push The other side's changes are lost
Rely on memory for post-release checks Things break when nobody is looking

Treating automation not as a way to fail less, but as a way to notice failures quickly and roll back makes priorities easier to set.

18. Checklist

Go through these when you build or change the release mechanism, and once a quarter.

  • The script starts with set -euo pipefail, and important steps exit 1 on failure
  • Before transferring, it checks that required files exist in the source
  • If --delete is used, planned deletions are reviewed in a dry run and capped with --max-delete
  • .env, uploads, server-generated data and logs are excluded
  • The application sits outside the web root, and APP_DEBUG is false in production
  • Dependencies are installed from lockfiles and resolved for the production PHP version
  • CSS and JS file names carry fingerprints, and HTML is revalidated every time
  • Schema changes are split into add, migrate, drop, with a backup before running
  • Only one production release runs at a time, and running ones are never cancelled
  • You can roll back to the previous release within a minute, and have actually tried it
  • Right after release, key pages, text, noindex and robots.txt are checked automatically

19. Summary

  • Deployment accidents happen across four stages, build, transfer, switch and after release, in nine types
  • The easiest to prevent and most damaging are unexpected deletions and overwritten config; prevent them with a dry run, exclusions, a deletion limit and a source check
  • Stop on failure. With set -euo pipefail and rsync's exit codes, never publish a half-finished state
  • Use fingerprinted file names and revalidated HTML to avoid stale pages and mixed versions
  • Database schema changes do not roll back with the files. Add, migrate, drop, with a backup first
  • Use release directories and a link switch so you can roll back within a minute
  • Automate the immediate post-release check; keep the next-day Search Console check with a person

Sources

Transfers and scripts

CI, repositories and secrets

Builds, dependencies and frameworks

Caching and search

Operating principles and incident reports

Security guidance and reporting

All were checked on 31 October 2026. Tool behaviour, prices and support dates change, so check the official documentation for the latest information before you start. Costs, effort and recovery times are DevNJ's working estimates and vary with the size of the site and the contract.

FAQ

What is the most common accident in automated website deployments?
On small sites, the two most common are using a transfer tool's delete option and wiping files that only exist in production, such as uploaded images or config files, and a build or transfer failing halfway while the script carries on and publishes a broken site. A dry run that lists planned deletions first, plus stopping whenever an exit code is not 0, prevents most of them.
Is it safer not to use rsync --delete?
Without it, files you meant to remove stay in production, which can cause bugs or leak information. The official rsync manual says --delete can be dangerous if used incorrectly and recommends a --dry-run first to see which files will be deleted. In practice, combine it with exclusions, a --max-delete limit and a check that the source directory is not empty.
How should I prepare for rollbacks?
Put each release in its own directory and switch only the link the web server uses. Rolling back is then just pointing the link at the previous release. The deployment tool Deployer keeps the last 10 releases by default. Database schema changes are not undone by switching the link back, so you also need a backup taken before the change and an order of changes that the old code can still run on.
Does a small business website need a staging environment?
Sites that take enquiries or orders, and sites that run PHP or a database, are safer with one. For a static site, preview builds for each pull request can do the job. Protect staging with a password so it does not appear in search results. If you hide it with noindex instead, check before each release that the setting has not been carried over to production.
What should I check after a release, and for how long?
Straight after the release, automatically check the response code and a fixed text string on 5 to 10 key pages. For the next 5 to 30 minutes, watch the error log and send a test form. The next day, check Search Console for new errors and 404s. Google advises keeping 503 responses to about 1 to 2 days, so do not leave a site down for long.
How much does it cost to make deployments safe?
As a DevNJ estimate: adding checks and stop conditions to an existing script takes 6 to 12 hours; adding CI builds and release switching takes 16 to 32 hours; adding staging and database safeguards takes 40 to 80 hours. Outsourced at €80 an hour, set-up costs range from €480 to €6,400 (about ¥84,000 to ¥1.12 million at ¥175 to the euro), with running costs of €0 to €80 a month.

Planning a site that has to work in both markets?

Get in touch

Contents