README

This package allows you to download, export and import your application's database backups.

Important

This package will not work if you have disabled "proc_open" in your PHP configuration.

Common usage scenarios

• Store your local database in a file
• Non-productive developer machines can download the live server database
• A central backup server can collect backups from multiple live servers

Feature set

• Download and optionally import databases from a server
• Import existing database files
• Export the local database to a file
• User authentication through Laravel Sanctum tokens
• Transport encryption using Sodium
• Laravel disk support

Supported databases

Protector supports the following databases:

MySQL is no longer officially supported, but the Protector still has capabilities to work with Laravel's mysql driver. If this should break in the future, feel free to submit a PR.

Note

• Source and destination databases are not validated. Make sure you run compatible software versions to prevent issues.
• Because of different dump formats, dumps will not able to be imported into a different database engine, e.g. a MariaDB dump will fail to be imported into PostgreSQL, and vice versa.

Notes

• Enabling Laravel Telescope will prevent remote files from being downloaded, as it opens and discards the HTTP stream!

Table of contents

• Usage
  • Export to file
  • Import
• Setup instructions
  • Setup for storing the local database
  • Setup for importing the database of a remote server
  • Setup for collecting backups from multiple servers
• Configuration
  • Disks
  • Dump metadata
• Development

Usage

Export to file

To save a copy of your local database, run

php artisan protector:export

To configure settings, such as the file name, you can either publish the config file, or set the according environment variables found in the ProtectorEnv class.

php artisan vendor:publish --tag=protector.config

For configuring

• the storage location, see the Disks section.
• the metadata appended to the dump file, see the Dump metadata section.

Download from remote

To store the newest remote dump without importing it, run

php artisan protector:download

To store and import in one step

php artisan protector:download --import

If you want to delete all files on the storage disk except the newly stored dump, run

php artisan protector:download --import --flush-storage

Each stored dump also has a matching metadata file with the .meta suffix (for example dump.sql.meta). The metadata file stores the same metadata object that is embedded in the SQL dump footer under meta.

Interactive import reads metadata from these metadata files. If a metadata file is missing, the dump can still be selected, and the import command will group it as an unknown connection.

Flushing will only delete files with existing .meta files, and will not delete files in directories.

Import

Run the following command for an interactive shell

php artisan protector:import

Importing a specific source

To download and import the server database in one go without storing the dump, run

php artisan protector:import --remote

protector:import cleans up the used disks after importing.

When used with other options, remote will serve as fallback behavior.

To import a specific database file that you downloaded earlier, run

php artisan protector:import --file=<relative path on Protector storage disk>

To import the latest existing database file, run

php artisan protector:import --latest

Options

If you want to run migrations after the import of the database file, run

php artisan protector:import --migrate

For automation, consider the force option to bypass user interaction.

php artisan protector:import --remote --migrate --force

To learn more about import options, run

php artisan protector:import --help

Setup instructions

Find below three common scenarios of usage. These are not mutually exclusive.

Setup for storing the local database

If you only want to store a copy of your local database to a disk, the setup is pretty straightforward.

Installing Protector in your local Laravel project

Install the package via composer.

composer require cybex/laravel-protector

Almost all config options can be set via environment variables. Take a look at the ProtectorEnv class for all available options.

You can optionally publish the Protector config to have more fine-grained control over config settings:

php artisan vendor:publish --tag=protector.config

Local usage

You can now use the artisan command to write a backup to the Protector storage folder.

php artisan protector:export

By default, the file will be stored in storage/private/protector and have a timestamp in the name. You can also specify the filepath.

You could also automate this by

• installing a cronjob on linux
• running it when you deploy to your server
• creating a Laravel Job and queueing it

php artisan protector:export --file="database.sql"

Setup for importing the database of a remote server

This package can run on both servers and client machines of the same software repository. You set up authorized developers on the server and give them the key for their local machine.

Installing Protector in your Laravel project

Install the package via composer.

composer require cybex/laravel-protector

In your User model class, add the following trait.

use Laravel\Sanctum\HasApiTokens;

class User extends Authenticatable
{
    use HasApiTokens;

    ...
}

Publish the Protector database migration and optionally modify it to work with your project.

php artisan vendor:publish --tag=protector.migrations

Publish the Laravel Sanctum migration, to make the personal_access_tokens table available.

php artisan vendor:publish --tag=sanctum-migrations

Run the migrations on the client and server repository.

php artisan migrate

You can use environment variables or optionally publish the Protector config to set options regarding the storage, access and transmission of the files.

php artisan vendor:publish --tag=protector.config

On the client machine

Run the following command to receive

• the public key to give to your server admin
• the private key to save in your .env file

php artisan protector:keys

Important

Do not give your private key to anyone and keep it protected at all times!

Your server admin will then give you the token and dump endpoint URL to save in your .env file.

PROTECTOR_CLIENT_AUTH_TOKEN=
PROTECTOR_CLIENT_DUMP_ENDPOINT_URL=

See Usage on how to import the remote database.

Note

Downloaded database dump files are stored unencrypted.

On the server

Make sure that the server is accessible to the client machine via HTTPS.

When one of your developers gives you their public key, you can authorize them with:

php artisan protector:token --publicKey=<public key> <user id>

You will receive the token and dump endpoint URL to give back to the developer, who has to save them in their .env file.

The developer can then download and import the server database on their own.

Setup for collecting backups from multiple servers

You can develop a custom client that can access and store remote server backups. The servers can be different Laravel projects that have the Protector package installed.

See the previous chapter on how to give your backup client access to all servers. The backup client will need an according user on each target server.

• All the backup users on the target servers will have the same public key from the client
• For each target server, the client will store the according url and token

See cybex-gmbh/collector for an example implementation.

Configuration

The protector.php config file sets initial settings for the Protector instance.

Generally, you should keep the Protector singleton instance as is. To create a new instance with different settings, use the ProtectorConfigurator class. For all available configuration options, take a look at the ProtectorConfiguratorContract.

For example, to configure a specific auth token and dump endpoint URL:

$protector = ProtectorConfigurator::setAuthToken($authToken)->setDumpEndpointUrl($dumpEndpointUrl)->createProtector();

Disks

There are two disks, which use the local driver by default:

• protector_local is used for temporary files which are deleted after use
  • writes to storage/app/private/protector_local by default
• protector_storage is used for storing dumps and their metadata files
  • writes to storage/app/private/protector by default

Important

The protector_local disk must be a local disk, as certain operations require a local filesystem, such as creating or importing a database dump.

Almost all operations go through the local disk by creating a local copy first, an exception to this is passing an absolute path to import operations, such as protector:import --file=/path/to/dump.sql

If you want to override the disk configuration, add the following to your config/filesystems.php file:

'protector_local' => [
    ...
],

'protector_storage' => [
    ...
],

You could for example use S3 for the storage disk.

Dump metadata

Customize the metadata appended to a dump by adding providers to the dump.metadata.providers array in your config/protector.php file:

'providers' => [
    \Cybex\Protector\Classes\Metadata\Providers\EnvMetadataProvider::class,
    \Cybex\Protector\Classes\Metadata\Providers\GitMetadataProvider::class,
    \Path\To\Your\CustomMetadataProvider::class,
],

Available metadata providers:

1. DatabaseMetadataProvider: Will always be appended. Adds general information about the dump, such as the database connection and dumped at date.
2. ProtectorMetadataProvider: Adds information about the settings set on the Protector's config.
3. EnvMetadataProvider: Adds information based on an .env value. The default .env key used for this is PROTECTOR_METADATA.
4. GitMetadataProvider: Adds information about the Git repository, such as the current branch and revision.
5. JsonMetadataProvider: Adds information from a JSON file. The default file path used for this is protector_metadata.json.

Note

You can create your own metadata providers by implementing the Cybex\Protector\Contracts\MetadataProvider interface. Duplicate provider keys will be merged in the final metadata array, so choose a unique key.

Tip

An example of using the JsonMetadataProvider would be to add custom metadata from a CI/CD pipeline. For example, in a GitHub Actions workflow, you could add a step that writes Git information to protector_metadata.json

- name: Protector Metadata
  shell: bash
  run: >
    jq -n \
      --arg repo ${{ github.repository }} \
      --arg branch ${{ github.ref_name }} \
      --arg revision ${{ github.sha }} \
      --arg buildDate "$(date --iso-8601=seconds --utc)" \
      '{gitRepo: $repo, gitBranch: $branch, gitRevision: $revision, buildDate: $buildDate}' > protector_metadata.json

Development

There is an example app with the Laravel Protector package installed.

The file structure in the container is as follows:

• /var/www: example app
• /var/package: Protector package

docker compose up -d

docker compose exec app shell

composer install

Note

We disable composer security checking for this package, as vulnerabilities would block the development. The project requiring our package should be responsible for evaluating possible vulnerabilities. For more information, see the composer documentation.

Specific to the example app, for demo data:

php artisan migrate:fresh --seed

Note

The example app uses the same database as the Unit tests, which might pollute the DB with data. For a reproducible environment, always run the above command before executing commands in the example app.

Testing

Run tests on the MariaDB database:

composer test

Run tests on the PostgreSQL database:

composer test-postgres

Run tests on the MySQL database:

Note

Running MySQL tests on the current alpine image will not work, as the MySQL CLI command is only an alias to mariadb and does not fully support the MySQL server.

If you need to run MySQL tests, use a different image. To start up the mysql server, use docker compose --profile mysql up -d

composer test-mysql

Test coverage

To generate coverage, you need to run the tests from the package directory.

cd ../package

composer test

Contributing

Please see CONTRIBUTING for details.

Security

If you discover any security-related issues, please email webdevelopment@cybex-online.com instead of using the issue tracker.

Credits

• Web Development team at Cybex GmbH - cybex-online.com
• Gael Connan
• Jörn Heusinger
• Fabian Holy
• Oliver Matla
• Marco Szulik
• All Contributors

License

The MIT License (MIT). Please see License File for more information.

Laravel Package Boilerplate

This package was generated using the Laravel Package Boilerplate.