On-premise Installation via VM image(s)¶
Test / PoC environment¶
SentinelTrails can be provided as a virtual appliance in Open Virtualization Format. The provided appliance is pre-configured and should be deployed with the virtualization platform used by the customer. Then the image needs configuring a few parameters:
If DCHCP is not available, configure IPs in
/etc/sysconfig/network-scripts/ifcfg-xxx
-BOOTPROTO=static
,IPADDRESS=<ip>
,GATEWAY=<gateway>
,NETMASK=<netmask>
. Then runservice network restart
.root.url
in/var/logsentinel/app.properties
should be set to the IP address or internal domain name of the machine.spring.mail.*
properties if email notifications are required. The application can work without that for test purposes.server.ssl.*
properties (see reference) should be configured in case the service is usedsecure.headers
should be set totrue
if TLS is used (either configured viaserver.ssl
properties or at a load balancer/reverse proxy put in front of the applicationjwt.secret
andhmac.key
should be set to new values for non-test environmentsAfter all changes are made, the SentinelTrails service should be restarted by running
service logsentinel restart
Production environment¶
In order to run successfully in production, SentinelTrails needs to be run in a cluster. The typical cluster setup includes the following nodes, all of which are provided in OVF:
2 applications nodes - all the configuration options for a Test / PoC environment apply here as well, plus additional configurations below
2 database nodes (cassandra) - cassandra nodes require minimal configuration to run in a cluster
3 elasticsearch nodes (optionally, 2 ES nodes can be configured, but that risks creating a split-brain scenario in case of failure of connectivity between the nodes)
load balancer node(s) - running nginx as a reverse proxy that forwards requets to the application nodes
Application node configuration¶
In addition to the base properties above, the following needs to be set:
hazelcast.nodes
- comma-separated list of the static IP addresses (or domain names) of the application nodescassandra.hosts
- comma-separated list of the static IP addresses (or domain names) of the database nodeselasticsearch.url
- comma-separated list of the URLs to the elasticsearch nodes. If DNS round-robin us used, this can be a single URLetherscan.key
- in case hashes are to be pushed to Ethereum, an Etherscan key should be configuredstorage.local.root
- we recommend setting up NFS shared directories and configuring them as a shared storage locations
Cassandra configuration¶
Cassandra needs the IP addresses of seed node(s) confiugred as in
/etc/cassandra/cassandra.yaml
JVM options should be configured in
/etc/cassandra/conf/jvm.options
depending on the available memory
We only work with Cassandra 3.11 (the latest version)
Elasticsearch configuration¶
Standard elasticsearch cluster configuration is required as described here.
Elasticsearch 6.x and 7.x are supported.
Supported Operating Systems¶
We support virtual machines with the following operating systems:
CentOS 7
CentOS 8
Ubuntu 20.04
Ubuntu 19.10
Ubuntu 18.04
Amazon Linux 2
Backup¶
Backup is ideally performed on the underlying storage. If needed, Cassandra backup tooling (Priam) can be configured by our team.
Monitoring¶
Different organizations have different tools available for monitoring certain aspects of their infrastructure. We don’t impose any tool and will assist the organization in creating the necessary rules with their tooling, including installing an agent, if required. Typical monitoring components are:
Free disk space - while you can configure retention periods for the data and it will automatically get deleted from the database and cassandra (and moved to the configured NFS storage), it’s important to be alerted if disk space is running out on either of the nodes.
CPU usage - prologned CPU usage over 70% may indicate that the existing CPUs are not sufficient to handle the load and a new or more preformant application node is needed
Free memory - all nodes are heavily RAM-dependent, so the current free memory should be monitored
Healthchecks - lightweight HTTP healthchecks can be performed against the
/login
URL. Additionally, an authenticated /healthcheck` is exposed for more detailed healthchecks, including database and elasticserach status