You can edit this page using wiki syntax for markup.

Article contents:== Outdated egg! This is an egg for CHICKEN 4, the unsupported old release. You're almost certainly looking for [[/eggref/5/ugarit-backend-s3|the CHICKEN 5 version of this egg]], if it exists. If it does not exist, there may be equivalent functionality provided by another egg; have a look at the [[https://wiki.call-cc.org/chicken-projects/egg-index-5.html|egg index]]. Otherwise, please consider porting this egg to the current version of CHICKEN. [[tags:egg]] == ugarit-backend-s3 [[toc:]] === Description This egg provides a [[ugarit|Ugarit]] storage backend for Amazon S3 and other stores with an S3-compatible API. === Requirements * [[ugarit]] * [[amazon-s3]] * [[openssl]] === Usage To make use of this egg, you'll first need an account for an object store with an S3-compatible service provider. This egg has been tested with PCextreme's [[https://www.pcextreme.nl/aurora/objects|Aurora Objects]] offering (which uses [[https://ceph.com/|Ceph]]) and it should of course also work with the original S3 storage from Amazon. If you use another provider and run into problems, please let me know. Next, you create two files. One is the Ugarit configuration for this backend (see the [[#recommendations|recommendations section]] below for more information about some choices): <enscript highlight="scheme"> (storage "sudo -u backup /usr/local/chickens/ugarit/bin/ugarit-backend-s3 /etc/ugarit/s3.conf") ;; Without sudo, it should just look something like this: ;;(storage "ugarit-backend-s3 /etc/ugarit/s3.conf") (cache "/etc/ugarit/s3-cache.sqlite") ;; You can generate these hashes with the following command: ;; `head -c 64 /dev/urandom | base64 -w0' (hash tiger "EsqAF7cTyrmo1cQjamjKo/nU5gENPaNtpcLJgn4OYbkLI+swCGl+Hddi5iDv7qijphvpJQsDI4WZOlvMouvVqQ==") ; requires the "tiger-hash" egg (encryption aes (32 "uE0QVQJHZboV120HxcO/8DB06oN6qNCXT8E01++oUaMlwiETi9PIvFIYcyiJaEIihn+e7hyMjRPq4O8sjXeuKw==")) ; requires the "aes" egg (compression deflate) ; requires the "z3" egg ;; Simple example to include only specific directories from /etc and ;; your homedir (the default is to include all files) (rule "/" (not (or (name "home") (name "etc"))) exclude) (rule "/home/user" (not (or (name "docs") (name ".emacs"))) exclude) </enscript> You will probably have already noticed the other file you'll have to create, which is referenced in the Ugarit config. This contains the S3-specific settings. This file is separated from the ugarit config, because the backend runs as a separate process, and we don't want to pass the secrets as a command line argument (that would expose it to prying eyes via {{ps}}). Here is what it should look like: <enscript highlight="scheme"> (access-key "ABCDEFGHIJ0123456789") (secret-key "my-secret-password") (bucket "my-ugarit-bucket") ;; Optional, to be used if you have a different ;; storage provider than Amazon: (base-hostname "o.auroraobjects.eu") </enscript> Finally, once you have set it up, you can make a backup like usual: # ugarit snapshot /etc/ugarit/host.conf my-host / That's it! You can of course explore the snapshot just as easily: $ ugarit explore /etc/ugarit/host.conf This will still perform the {{sudo}} command to run the backend, so this may ask you for your sudo password. You can also run it as root (which is recommended when you need to restore something, so the permissions can be set correctly). === Configuration options The following configuration options are supported in the configuration file that's specific to the S3 backend. The following options are all '''required''': (access-key STRING) (secret-key STRING) (bucket STRING) These are the credentials to use when connecting to your S3 store and the bucket in which the data is kept. Note that the bucket must already exist before you can make a snapshot! The following options are '''optional''': (base-hostname STRING) The base hostname to use. Defaults to {{"s3.amazonaws.com"}}. By default, the bucket name will be prepended to this host name, separated with a dot. use-bucket-path When this occurs in the file (without parentheses), the bucket is not prepended to the {{base-hostname}}, but it is used as the first component of the path (for compatibility with certain kinds of [[http://docs.ceph.com/docs/master/radosgw/s3/commons/|Ceph setups]]). === Security recommendations ==== Encrypt your data Rule number 1: When backing up to a cloud service like S3, '''always''' use encryption (e.g. the {{aes}} option in your ugarit config)! You never know when the provider may get hacked, old disks might get discarded and scooped up, your buckets might accidentally be world-readable (a [[https://www.bleepingcomputer.com/news/security/7-percent-of-all-amazon-s3-servers-are-exposed-explaining-recent-surge-of-data-leaks/|bigger problem than you might think]]) or some agency might be poking its nose where it doesn't belong. ==== Salt your data As per the Ugarit documentation, use the {{(hash)}} option to provide a salt, so that blocks with the same content can't be easily detected across different vaults. The only disadvantage of this is that this will use more disk space if you use the same S3 bucket for multiple hosts - something which is explicity '''not''' recommended. If you use multiple vaults on the ''same'' host, feel free to use the same salt between those vaults to allow sharing of identical blocks. ==== Use separate S3 accounts for each machine In a worst case scenario, your machine might get hacked. If this happens, the attacker might find your Ugarit configuration, which includes S3 credentials. Using these credentials, the attacker might decide to wipe or overwrite/corrupt all data from your S3 bucket. That's why it's important to use a different S3 account for every machine. If one gets hacked, the backups from the other machines cannot also be corrupted. There is nothing Ugarit can do against this: if you back up to a directory, that can also be erased. To mitigate this, you could enable S3 versioning to make it behave as an append-only file system, but note that that will also drive up your price, and the current backend has no support for reading from older S3 versions. Neither does {{s3cmd}}, so you'll need to determine if it's worthwhile. ==== Use privilege separation for the backend Using {{sudo}} (or {{doas}}, etc) to run the backend is strongly recommended. This will avoid having to make network connections as root. Use an account that has as few privileges as possible; it will only need access to the S3 configuration file, the OpenSSL default certificate store, and to make network connections. ==== Do not use sensitive info for your tag names Tag names are used in the (unencrypted) file names in the backing store, so you should avoid using tag names that might be considered sensitive information. Note that this goes for every backend, as this is part of the Ugarit design. === Authors [[/users/peter-bex|Peter Bex]] === Changelog * 0.2 - Avoid crashing when there's an inconsistency in the state between the {{.type}} and {{.data}} files (the {{.type}} file should be there only when the {{.data}} file is also there, but if the process shut down in an unclean way, you can end with a partial state) * 0.1.1 - Add missing [[openssl]] dependency * 0.1 - Initial release === Repository [[http://code.more-magic.net/ugarit-backend-s3]] === License All code in this egg is explicitly placed in the public domain. You may do whatever you want with it. Description of your changes:

I would like to authenticate

Spam control

What do you get when you subtract 6 from 23?