Threatbus

<h1 align="center"> Threat Bus </h1> <h4 align="center">

A threat intelligence dissemination layer for open-source security tools.

Getting Started —
Contributing Guidelines —
Writing Plugins —
License —
Documentation

</h4> <div align="center">

</div>

Key Features

• Connect Open-Source Security Tools: Threat Bus is a pub-sub broker for
  threat intelligence data. With Threat Bus you can seamlessly integrate
  threat intel platforms like OpenCTI or MISP with detection
  tools and databases like Zeek or VAST.

• Native STIX-2: Threat Bus transports indicators and sightings encoded as
  per the STIX-2
  open format specification.

• Plugin-based Architecture: The project is plugin-based and can be extended
  easily. Read about the different plugin types and
  how to write your own.
  We welcome contributions to adopt new open source tools!

• Official Plugins: We maintain many plugins right in the official Threat
  Bus repository. Check out our integrations for MISP, Zeek,
  CIFv3, and generally apps that connect via ZeroMQ, like
  vast-threatbus and our
  OpenCTI connector.

• Snapshotting: The snapshot feature allows subscribers to directly request
  threat intelligence data for a certain time range from other applications.
  Threat Bus handles the point-to-point communication of all involved apps.

Getting Started

The config.yaml.example file provides a working configuration for Threat Bus
with all existing application plugins enabled together with the RabbitMQ
backbone.

The following example shows how to connect Zeek via Threat Bus. There
are more integrations available, so make sure to check out all
Threat Bus projects on PyPI.

The example assumes that threatbus is available in your PATH. See the
section on Installation below for more information on how to
get there.

Start Threat Bus

shCopy
threatbus

Start with a specially named config file

The config.yaml.example file in this directory gives an overview of
the available config keys and their default values.

shCopy
threatbus -c /path/to/your/special-config.yaml

Environment variables take precedence over config file values. Prefix
everything with THREATBUS_

shCopy
export THREATBUS_LOGGING__CONSOLE=true
threatbus -c /path/to/your/special-config.yaml

Note that you must use a double underscores __ in your env to refer to nested
config variables.

Start Zeek as Threat Bus app

shCopy
zeek -i <INTERFACE> -C ./apps/zeek/threatbus.zeek

Start Zeek and request a snapshot

shCopy
zeek -i <INTERFACE> -C ./apps/zeek/threatbus.zeek "Tenzir::snapshot_intel=30 days"

Threat Bus also ships as pre-built Docker image and is available on
Docker Hub.

Use the Threat Bus Docker container

shCopy
docker run tenzir/threatbus:latest --help

Start Threat Bus container with a custom config file

shCopy
docker run -p 47661:47661 -v $PWD/my-custom-config.yaml:/opt/tenzir/threatbus/my-custom-config.yaml tenzir/threatbus:latest -c my-custom-config.yaml

Tip: Threat Bus checks for config files with default names. If you mount your
config file to /opt/tenzir/threatbus/config.yaml, you can start the
application without specifying the config file location with the -c parameter.

Installation

Install threatbus and all plugins that you require. Optionally, use a virtual
environment.

Note that Threat Bus requires at least Python 3.7+, earlier versions are not supported.

virtualenv venv                       # optional
source venv/bin/activate              # optional
pip install threatbus
pip install threatbus-inmem           # inmemory backbone plugin
pip install threatbus-rabbitmq        # RabbitMQ backbone plugin
pip install threatbus-misp[zmq]       # MISP application plugin
pip install threatbus-zeek            # Zeek application plugin
pip install threatbus-zmq             # ZeroMQ application plugin
pip install threatbus-<plugin_name>

Testing

Use the Makefile to run unit and integration tests.

make unit-tests
make integration-tests

The integration tests require a local Zeek and
Docker installation.

Development

Setup a virtual environment and install threatbus and some plugins with the
in development mode:

virtualenv venv
source venv/bin/activate
make dev-mode

Configuration & Extension

A plugin must define a setup.py. Whenever a plugin is installed, you have to
add a corresponding configuration section to threatbus' config.yaml. That
section has to be named after the name in the entrypoint declaration of the
plugin's setup.py file.

Please adhere to the plugin naming conventions
and always prefix your plugin name with threatbus-.

Plugins can either be apps or backbones. Application plugins (apps) add new
functionality to Threat Bus and allow communication to a specific app and/or
via a specific protocol (e.g., ZeroMQ or Zeek/broker). Backbone plugins add a
new storage and distribution backend to Threat Bus (e.g., in-memory or
RabbitMQ).

Example:

• plugin folder structure:

  shCopy
  plugins
  ├── apps
  |   └── threatbus-myapp
  │       ├── setup.py
  |       └── threatbus_myapp.py
  └── backbones
      └── threatbus-inmem
          ├── setup.py
          └── threatbus_inmem.py

• setup.py

  pyCopy
  from setuptools import setup
  setup(
    name="threatbus-myapp",
    install_requires="threatbus",
    entry_points={"threatbus.app": ["myapp = threatbus_myapp"]},
    py_modules=["threatbus_myapp"],
  )

• config.yaml entry for threatbus

  yamlCopy
  ...
  plugins:
    apps:
      myapp:
      ...

Threat Bus API

Plugins specifications are available in threatbus/appspecs.py and
threatbus/backbonespecs.py, respectively. For any plugin, you should at least
implement the run function.

App plugins are provided two callback functions to use for subscription
management. Internally, Threat Bus will propagate subscription requests to all
installed backbone plugins.

The subscription callback allows applications to request an optional snapshot
time delta. Threat Bus will forward snapshot requests to all those apps that
have implemented the snapshot feature (see threatbus/appspecs.py).

Implementation

Please use the
StoppableWorker
base class to model your plugin's busy work. Plugins should never block the main
thread of the application. Implementing that class also facilitates a graceful
shutdown.

All officially maintained Threat Bus plugins implement StoppableWorker. Refer
to any of the existing plugins for an example.

License

Threat Bus comes with a 3-clause BSD license.