Documentation cleanup #5

Manually merged
armin merged 10 commits from develop into master 2020-03-05 05:03:52 +00:00
16 changed files with 314 additions and 133 deletions

View file

@ -24,15 +24,26 @@ steps:
image: clux/muslrust
commands:
- cargo build --release
- strip target/x86_64-unknown-linux-musl/release/coffer-server
- strip target/x86_64-unknown-linux-musl/release/coffer-client
- strip target/x86_64-unknown-linux-musl/release/coffer-companion
- mkdir coffer-${DRONE_TAG}-x86_64-musl
- mv target/x86_64-unknown-linux-musl/release/coffer-server \
target/x86_64-unknown-linux-musl/release/coffer-client \
target/x86_64-unknown-linux-musl/release/coffer-companion \
coffer-${DRONE_TAG}-x86_64-musl
- strip coffer-${DRONE_TAG}-x86_64-musl/coffer-server
- strip coffer-${DRONE_TAG}-x86_64-musl/coffer-client
- strip coffer-${DRONE_TAG}-x86_64-musl/coffer-companion
- name: package
image: alpine
commands:
- tar cjf coffer-${DRONE_TAG}-x86_64-musl.tar.bz2 target/x86_64-unknown-linux-musl/release/coffer-server target/x86_64-unknown-linux-musl/release/coffer-client target/x86_64-unknown-linux-musl/release/coffer-companion
- tar czf coffer-${DRONE_TAG}-x86_64-musl.tar.gz target/x86_64-unknown-linux-musl/release/coffer-server target/x86_64-unknown-linux-musl/release/coffer-client target/x86_64-unknown-linux-musl/release/coffer-companion
- tar cjf coffer-${DRONE_TAG}-x86_64-musl.tar.bz2 \
coffer-${DRONE_TAG}-x86_64-musl/coffer-server \
coffer-${DRONE_TAG}-x86_64-musl/coffer-client \
coffer-${DRONE_TAG}-x86_64-musl/coffer-companion
- tar czf coffer-${DRONE_TAG}-x86_64-musl.tar.gz \
coffer-${DRONE_TAG}-x86_64-musl/coffer-server \
coffer-${DRONE_TAG}-x86_64-musl/coffer-client \
coffer-${DRONE_TAG}-x86_64-musl/coffer-companion
- name: publish
image: plugins/gitea-release

2
.gitattributes vendored
View file

@ -1,2 +1,4 @@
*.enc filter=lfs diff=lfs merge=lfs -text
*.cert filter=lfs diff=lfs merge=lfs -text
overview.png filter=lfs diff=lfs merge=lfs -text
overview.svg filter=lfs diff=lfs merge=lfs -text

2
Cargo.lock generated
View file

@ -88,9 +88,7 @@ dependencies = [
"exec 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
"futures 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
"log 0.4.8 (registry+https://github.com/rust-lang/crates.io-index)",
"serde 1.0.102 (registry+https://github.com/rust-lang/crates.io-index)",
"serde_cbor 0.10.2 (registry+https://github.com/rust-lang/crates.io-index)",
"serde_yaml 0.8.11 (registry+https://github.com/rust-lang/crates.io-index)",
"structopt 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)",
]

View file

@ -1 +1,77 @@
# Coffer
[![Build Status](https://drone.friedl.net/api/badges/incubator/coffer/status.svg?ref=refs/heads/develop)](https://drone.friedl.net/incubator/coffer)
Coffer is a collection of tools for the simple but secure management of
application configuration.
It is meant to be flexible and simple, hence does not assume much about your
environment. Especially, you don't need a kubernetes cluster for running coffer.
Coffer runs directly on your server, just as well as in a containerized setup or
a full kubernetes cluster. The only thing Coffer needs is a TCP connection
between the `coffer-server` (securely holding your configuration) and the
`coffer-client` (retrieving configuration and setting up your application).
## Overview
![](overview.png)
## The Parts of Coffer
Coffer is split into 3 binaries and a supporting library:
* `coffer-server`: The `coffer-server` securely stores configuration data and
hands them out to `coffer-clients` upon request
* `coffer-client`: A `coffer-client` requests configuration data from a
`coffer-server`, configures the application and may also start it up
* `coffer-companion`: A helper for generating certificates and encrypting
configuration data
* `coffer-common`: A common library for all binaries containing common
cryptographic operations, protocol code and interface definitions
## Security
Coffer does not rely on a secure connection or any specifics of the environment.
Instead security is provided by a basic public key cryptography scheme. This
gives you, the user, the flexibility to set up your ecosystem according to your
own security needs.
### Certificates
Certificates in coffer are the just a keypair consisting of a public and private
key as used by public key cryptography. So, basically, a certificate is nothing
more than a tuple of two primes.
Certificates in coffer can be generated by the `coffer-companion`. Every
`coffer-server` and `coffer-client` need their own certificate. Security
in coffer squarely depends on these certificates being kept secret.
### Configuration
Configuration can be written in [toml](https://github.com/toml-lang/toml)
format. It is secured by encrypting it with the public key of the
`coffer-server`. This can be done by invoking the `coffer-companion`.
Encrypted configuration can be conveniently stored in VCS (e.g. via git-lfs)
with your application. As long as the server certificate stays private.
### Communication
The communication between a `coffer-server` and a `coffer-client` is not and
does not need to be secured. A `coffer-server` associates configuration data
with the public keys of `coffer-clients`. Upon request of configuration data the
server sends back the configuration encrypted with the clients public key. Only
a client in posession of the corresponding private key can read the response.
### Trust Anchors
It is worth mentioning some things about trust anchors. Every cryptography
scheme, no matter how sophisticated, needs, at some point, something that can be
trusted. In case of HTTPS this is provided by central certificate authorities.
In case of the encrypted letters to your best friend this might be a pre-shared
password transmitted over phone in 1972.
From a coffer perspective keys can be trusted. That is, coffer assumes that
certificates are distributed and kept secret according to your threat model. An
attacker in control of a certificate can steal secret configuration!
Coffer does not assume a trust anchor for you. Instead, you are free to choose
your own trust anchor according. In a simple personal server setup this might
mean just distributing certificates by hand. In a more complex, corporate
environment you may want to set up a secure, central authority.
Trust anchors are a trade-off between convenience, complexity and security.
Coffer lets _you_ choose where along these axis you put your trust anchor.

View file

@ -1,27 +0,0 @@
#+TODO: TODO NEXT DONE
* General
** TODO Add a license
** TODO Better communication protocol
** TODO Add tests
** TODO Readme
* Coffer Server
** TODO Double delete if files not kept
** TODO Add secrets on-the-fly
** TODO Store secrets in secure memory
- Not persisted
- Nulled out
- Optional encrypted
* Coffer Client
** DONE Set environment variables
CLOSED: [2019-11-27 Wed 22:51]
** TODO Send key requests encrypted/signed
** TODO Secure Communication
* Coffer Companion
** TODO Add Subcommands
- [X] Generate certificate
- [ ] Seal secrets with certificate
- [ ] Open secrets with certificate
- [ ] Generate trampolin sh from dockerfile
* Docker
** TODO Create Dockerfile for server

View file

@ -12,8 +12,6 @@ log = "0.4"
env_logger="0.7"
structopt = "0.3"
# Communication
serde = { version = "1.0", features = ["derive"]}
serde_yaml = "0.8"
serde_cbor = "0.10.2"
# Executing subcommand
exec = "0.3.1"

View file

@ -1,20 +1,28 @@
//! # Coffer client
//!
//! Retrieve a secret shard from a `coffer-server`. Secrets in the shard are set
//! as environment variables for the spawned subcommand `cmd`.
#[allow(unused_imports)]
use log::{debug, error, info, trace, warn};
use env_logger;
use structopt::StructOpt;
use std:: {
net::{SocketAddr, TcpStream},
net::TcpStream,
error::Error,
path::PathBuf,
io::{Write, Read},
convert::{TryInto, TryFrom}
};
use coffer_common::certificate::Certificate;
use coffer_common::coffer::{CofferShard, CofferValue};
use coffer_common::{
coffer::{CofferShard, CofferValue},
certificate::Certificate
};
use structopt::StructOpt;
/// Client for setting up the environment from coffer server secrets
#[derive(StructOpt, Debug)]
struct Args {
/// Address of the coffer server
@ -75,6 +83,8 @@ fn main() -> Result<(), Box<dyn Error>> {
Err("Could not spawn sub-command".into())
}
/// Replaces the `coffer-client` process image with
/// the subcommand `cmd` with `args`
fn reap_coffer(cmd: &str, args: &[String]) {
let mut cmd = exec::Command::new(cmd);

View file

@ -1,18 +1,27 @@
//! Common certificate handling and encryption
//! A keypair container providing functionality for signing, encryption and
//! decryption
//!
//! # Base libraries
//! The cryptographic operations exposed by this module are based on the
//! [NaCl](http://nacl.cr.yp.to/) fork [libsodium](https://libsodium.org) as
//! exposed by the rust bindings [sodiumoxide](https://crates.io/crates/sodiumoxide).
//!
#[allow(unused_imports)]
use log::{debug, error, info, trace, warn};
use std::path::Path;
use std::io::BufReader;
use std::fs::File;
use std::fmt::{Debug, Formatter};
use std::{
path::Path,
io::BufReader,
fs::File,
};
#[allow(unused_imports)]
use std::ops::Deref; // we use this but rustc doesn't know
use quick_error::quick_error;
use seckey::SecKey;
use sodiumoxide::crypto::box_;
use sodiumoxide::crypto::sealedbox;
use sodiumoxide::crypto::{box_, sealedbox};
use serde::{Serialize, Deserialize};
use serde_cbor;
@ -25,22 +34,51 @@ quick_error! {
Io(err: std::io::Error) {
from()
}
SecKey
SecKey {
from(CertificateInner)
}
Crypto
}
}
/// A secure container for certificates
/// A secure container for a keypair
///
/// # Certificate
/// Secure means a best effort approach to:
/// - Prevent swapping memory to disk
/// - Zeroing out memory upon dropping
/// - Prevent other processes and buffer overflows to access the secure memory
/// area
///
/// A certificate consists of a public and a private key in a secure memory
/// area. With a certificate data sealed and opened.
/// These guarantees are currently *not* reliable. If you threat model contains
/// targeted attacks against coffer memory, additional precautions have to be
/// taken.
pub struct Certificate {
inner: SecKey<CertificateInner>
}
// The SecKeyReadGuard prevents convenience methods for handing out references
// to private/public keys (reference outlives SecKeyReadGuard). Hence below
// macros are shortcut projections that can be used after a read guard is
// created
// Get the public key
macro_rules! pk {
($cert:ident) => {
&$cert.inner.read().public_key
};
}
// Get the private key
macro_rules! sk {
($cert:ident) => {
&$cert.inner.read().private_key
};
}
// Certificate and its inner SecKey own their
// raw pointer without any thread local behaviour
unsafe impl Send for Certificate {}
// After initialization, certificate is read-only
unsafe impl Sync for Certificate {}
#[derive(Serialize, Deserialize)]
@ -49,13 +87,8 @@ struct CertificateInner {
private_key: box_::SecretKey
}
impl Debug for CertificateInner {
fn fmt(&self, fmt: &mut Formatter<'_>) -> std::fmt::Result {
write!(fmt, "<Certificate Hidden>")
}
}
impl Certificate {
/// Initialize with a generated keypair
pub fn new() -> Result<Certificate, CertificateError> {
debug!{"Generating new certificate"}
let (public_key, private_key) = box_::gen_keypair();
@ -66,6 +99,7 @@ impl Certificate {
Ok(Certificate{inner})
}
/// Initialize from a serialized certificate in [cbor](https://cbor.io/) format
pub fn new_from_cbor<T: AsRef<Path>>(path: T) -> Result<Certificate, CertificateError> {
debug!{"Reading certificate from {}", path.as_ref().display()}
let f = File::open(path)?;
@ -76,40 +110,35 @@ impl Certificate {
Ok(Certificate{inner})
}
/// Serialize a certificate to a file in [cbor](https://cbor.io/) format
#[cfg(feature = "export")]
pub fn to_cbor(&self) -> Result<Vec<u8>, CertificateError> {
let inner_cert = &*self.inner.read();
let cbor = serde_cbor::to_vec(inner_cert)?;
let read_guard = self.inner.read();
let cbor = serde_cbor::to_vec(read_guard.deref())?;
Ok(cbor)
}
/// Clone the bytes of the public key
pub fn public_key(&self) -> Vec<u8> {
self.inner.read().public_key.as_ref().to_owned()
pk!(self).as_ref().to_owned()
}
/// Clone the bytes of the private key
#[cfg(feature = "export")]
pub fn secret_key(&self) -> Vec<u8> {
self.inner.read().private_key.as_ref().to_owned()
sk!(self).as_ref().to_owned()
}
/// Open a [sealed box](https://download.libsodium.org/doc/public-key_cryptography/sealed_boxes)
pub fn open(&self, c: &[u8]) -> Result<Vec<u8>, CertificateError> {
let pk = &self.inner.read().public_key;
let sk = &self.inner.read().private_key;
sealedbox::open(c, pk, sk)
sealedbox::open(c, pk!{self}, sk!{self})
.map_err(|_| CertificateError::Crypto)
}
/// Seal a message in a [sealed box](https://download.libsodium.org/doc/public-key_cryptography/sealed_boxes)
pub fn seal(&self, message: &[u8]) -> Result<Vec<u8>, CertificateError> {
let pk = &self.inner.read().public_key;
Ok(sealedbox::seal(message, pk))
}
}
impl <T: AsRef<Path>> From<T> for Certificate {
fn from(path: T) -> Self {
Certificate::new_from_cbor(&path)
.expect(&format!{"Could not read certificate from {}", path.as_ref().display()})
Ok(sealedbox::seal(message, pk!{self}))
}
}

View file

@ -1,17 +1,85 @@
//! A storage container for client data
//!
//! Coffer supports separated client data by `CofferShard`s. Content of a shard
//! is a key-value store with typed values.
//!
//! # Coffer files
//! A `Coffer` can be read from a [toml](https://github.com/toml-lang/)
//! file in a specific format.
//!
//! ## Shards
//! A `CofferShard` is identified by a toml table with a field `id` containing
//! the unique shard id.
//!
//! Tables can be nested for grouping shards together. The grouping is not
//! necessarily reflected in the deserialized `Coffer`, as shards can be
//! uniquely identified by their id.
//!
//! Shards (tables with an id) cannot be nested.
//!
//! A simple shard with no data
//! ```toml
//! [app]
//! id = "1"
//! ```
//!
//! Grouped shard
//! ```toml
//! [app]
//! [app.frontend]
//! id = "1"
//!
//! [app.backend]
//! id = "2"
//! ```
//!
//! Nested shards (invalid)
//! ```toml
//! [app]
//! id = "1" # app is a shard since it has an id
//! [app.frontend] # invalid, can't nest shards inside other shards
//! id = "2"
//! ```
//!
//! ## Values
//! Shards can contain a subset of toml values. The currently supported toml
//! values are:
//! - [String](https://github.com/toml-lang/toml#user-content-string)
//! - [Integer](https://github.com/toml-lang/toml#user-content-integer)
//! - [Float](https://github.com/toml-lang/toml#user-content-float)
//! - [Boolean](https://github.com/toml-lang/toml#user-content-boolean)
//!
//! ## Example
//! ```toml
//! [app]
//! [app.frontend]
//! id = "1"
//! password = "admin"
//! font_size = 1.4
//!
//! [app.backend]
//! id = "2"
//! cors = true
//!
//! [database]
//! id = "0"
//! user = "root"
//! passwort = "toor"
//! ```
//!
#[allow(unused_imports)]
use log::{debug, error, info, trace, warn};
use std::fmt::Debug;
use std::path::Path;
use std::fs::File;
use std::io::{BufReader, Read};
use toml::Value as TomlValue;
use serde::{Serialize, Deserialize};
use std::{
fmt::Debug,
fs::File,
io::{BufReader, Read},
path::Path,
};
use quick_error::quick_error;
use toml::Value as TomlValue;
use serde::{Serialize, Deserialize};
quick_error! {
#[derive(Debug)]
@ -28,7 +96,7 @@ quick_error! {
pub type CofferResult<T> = Result<T, CofferError>;
/// Values supported by a `Coffer`
/// Values supported by `Coffer`
#[derive(Clone, Debug, Serialize, Deserialize)]
pub enum CofferValue {
/// A UTF-8 encoded string
@ -37,35 +105,38 @@ pub enum CofferValue {
Integer(i32),
/// A 32-bit float
Float(f32),
// A boolean
/// A boolean value
Boolean(bool)
}
/// A `CofferKey` defining the shard and the key into the kv-store
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq, Hash)]
pub struct CofferKey {
pub shard: String,
pub key: String
}
#[derive(Debug, Serialize, Deserialize)]
/// A key-value store for client data
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CofferShard(pub Vec<(String, CofferValue)>);
/// Interface for interacting with a `Coffer`
/// Trait for `Coffer`
pub trait Coffer {
/// Put `value` at `path`. Errors if there is already a value at `path`.
/// Put `value` at `key`. Errors if there is already a value for `key`.
fn put(&mut self, key: CofferKey, value: CofferValue) -> CofferResult<()>;
/// Push `value` to `path`. Replaces existing values.
/// Push `value` to `key`. Replaces existing values.
fn push(&mut self, key: CofferKey, value: CofferValue);
/// Retrieve `value` at path. Errors if there is no `value` at path.
fn get(&self, key: &CofferKey) -> CofferResult<CofferValue>;
/// Retrieve `value` at path. `None` if there is no `value` for `key`.
fn get(&self, key: &CofferKey) -> Option<CofferValue>;
/// Retrieve `value` at path. Errors if there is no `value` at path.
fn get_shard<T>(&self, shard: T) -> CofferResult<CofferShard>
/// Retrieve an entire shard. `None` if there is no `CofferShard` for `shard`.
fn get_shard<T>(&self, shard: T) -> Option<CofferShard>
where T: AsRef<str>;
fn from_toml_file(toml_path: &Path) -> Self
/// Deserializes a `Coffer` from a toml file
fn from_toml_path(toml_path: &Path) -> Self
where Self: Coffer + Default
{
// read the secrets file into a temporary string
@ -135,4 +206,5 @@ pub trait Coffer {
self.put(CofferKey{shard, key}, value).unwrap();
}
}
}

View file

@ -1,3 +1,4 @@
//! Certificate of the keyring owner plus known and trusted public keys
#[allow(unused_imports)]
use log::{debug, error, info, trace, warn};
@ -36,28 +37,36 @@ quick_error! {
}
}
/// Keyring container
///
/// A keyring constists of the owner's certificate and
/// the known and trusted public keys of the keyring owner
pub struct Keyring {
certificate: Certificate,
known_keys: HashMap<Vec<u8>, box_::PublicKey>
}
impl Keyring {
/// Create a new keyring
pub fn new(certificate: Certificate) -> Keyring {
Keyring {
certificate: certificate,
certificate,
known_keys: HashMap::new()
}
}
/// Deserialize a keyring from a file in [cbor](https://cbor.io) format
pub fn new_from_path<T>(certificate_path: T) -> Keyring
where T: AsRef<Path>
{
Keyring {
certificate: Certificate::from(certificate_path),
certificate: Certificate::new_from_cbor(certificate_path).unwrap(),
known_keys: HashMap::new()
}
}
/// Add the table ids of a `Coffer` in toml format as known keys to the keyring
// TODO: This needs to be refactored. Keyring shouldn't be that tightly bound to coffer format
pub fn add_known_keys_toml(&mut self, toml: &str) -> Result<(), KeyringError> {
// parse the string into a toml Table
let clients: toml::value::Table = match toml.parse::<TomlValue>().unwrap() {
@ -100,11 +109,21 @@ impl Keyring {
Ok(())
}
/// Open a sealed message with the keyring owner's certificate
pub fn open(&self, message: &[u8]) -> Result<Vec<u8>, KeyringError> {
self.certificate.open(message)
.map_err(KeyringError::from)
}
/// Seal a message for a client in the keyring
// TODO: Does this make sense?
// - What is a client in context of a keyring?
// - Why do we need to store trusted public keys in a keyring and not just
// encrypt for a pub key. Sealing does not need certificate.
// => We need authenticated encryption here. Sealed boxes can be tampered with by a MITM
// https://download.libsodium.org/doc/public-key_cryptography/authenticated_encryption
// I.e. a client could retrieve forged secrets by a attacker-controlled server, even if the
// real server's certificate was not exposed.
pub fn seal(&self, client: &[u8], message: &[u8]) -> Result<Vec<u8>, KeyringError> {
let client_key = self.known_keys.get(client)
.ok_or(KeyringError::UnkownClientKey)?;

View file

@ -1,16 +1,5 @@
//! Common base for coffer binaries
#[allow(unused_imports)]
use log::{debug, error, info, trace, warn};
//! Common traits and function for coffer implementations
pub mod certificate;
pub mod coffer;
pub mod keyring;
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}

View file

@ -10,7 +10,7 @@ pub fn generate_key(out: PathBuf) {
let cert = certificate.to_cbor().unwrap();
let mut writer = File::create(&out)
.expect(&format!{"Could not create out file {}", &out.display()});
.unwrap_or_else(|_| panic!{"Could not create out file {}", &out.display()});
writer.write_all(&cert).unwrap();
}

View file

@ -7,7 +7,7 @@ use std::sync::RwLock;
use std::sync::RwLockReadGuard;
use std::sync::RwLockWriteGuard;
use std::collections::HashMap;
use std::collections::hash_map::{HashMap, Entry};
use coffer_common::coffer::*;
@ -29,13 +29,15 @@ impl CofferMap {
}
impl Coffer for CofferMap {
fn put(&mut self, key: CofferKey, value: CofferValue) -> CofferResult<()> {
fn put(&mut self, key: CofferKey, value: CofferValue) -> CofferResult<()> {
let mut lock = self.write();
match lock.get_mut(&key.shard) {
Some(shard) => {
if shard.contains_key(&key.key) { Err(CofferError::Msg("Key exists")) }
else { shard.insert(key.key, value); Ok(()) }
match shard.entry(key.key) {
Entry::Occupied(_) => Err(CofferError::Msg("Key exists")),
Entry::Vacant(v) => { v.insert(value); Ok(()) }
}
}
None => {
lock.insert(key.shard.clone(), HashMap::new());
@ -59,33 +61,29 @@ impl Coffer for CofferMap {
}
}
fn get(&self, key: &CofferKey) -> CofferResult<CofferValue> {
fn get(&self, key: &CofferKey) -> Option<CofferValue> {
let lock = self.read();
let res = lock.get(&key.shard)
lock.get(&key.shard)
.and_then( |shard| { shard.get(&key.key) } )
.ok_or(CofferError::Msg("Key not found"))?;
Ok(res.clone())
.map(|o| o.clone())
}
fn get_shard<T>(&self, shard: T) -> CofferResult<CofferShard>
fn get_shard<T>(&self, shard: T) -> Option<CofferShard>
where T: AsRef<str>
{
let lock = self.read();
debug!{"Coffer {:?}", *lock}
let coffer_shard = lock.get(shard.as_ref())
.ok_or(CofferError::Msg("Shard not found"))?;
let map_to_vec = |map: &HashMap<String, CofferValue>| {
map.iter()
.map(|(k,v)| (k.clone(), v.clone()))
.collect::<Vec<(String, CofferValue)>>()
};
let mut res = CofferShard(Vec::new());
for (k,v) in coffer_shard {
res.0.push((k.clone(), v.clone()));
}
Ok(res)
lock.get(shard.as_ref())
.and_then(|s| Some(CofferShard(map_to_vec(s))))
}
}

View file

@ -59,7 +59,7 @@ where C: Coffer + Send + Sync + 'static
debug!{"Binding to socket {:?}", socket}
let mut listener = TcpListener::bind(socket).await
.expect(format!{"Could not bind to socket {}", socket}.as_str());
.expect("Could not bind to socket");
let server = async move {
let mut incoming = listener.incoming();

BIN
overview.png (Stored with Git LFS) Normal file

Binary file not shown.

BIN
overview.svg (Stored with Git LFS) Normal file

Binary file not shown.