Operational FAQ

Administration:

Can an existing sandfly-setup folder be moved to a different directory?

  • Yes, this can be done without the need to reinstall the software. To perform the move first ensure that the Task Queue is empty, shut down Sandfly with the shutdown_sandfly.sh script, then move the entire sandfly-setup folder to its new location, and finally restart Sandfly via the appropriate node or server start script.

Credentials:

Can more than one credential be attached to a given set of assets?

  • At present it is not possible to assign more than one credential to a host. This feature has been requested in the past. The complicating factor is that we do not want to activate any kind of auto banning that thinks that Sandfly is brute forcing. If we try 2-3 attempts that may be ok, but more than that and you use something like fail2ban then our service could become blocked.

Docker:

How to get a complete log from a Sandfly docker container?

  • From a shell on either a Sandfly server or node host, use the following commands:
  • docker ps      # To obtain the id of the target container
docker logs #CONTAINER_ID# &> /tmp/sandfly.log

Can Sandfly scan inside containers?

  • Currently Sandfly can scan the host / infrastructure that runs the containers and from there have full visibility into all of the containers running on that host. Our focus and testing has been on the underlying infrastructure that is running the containers, not container-at-a-time.
  • If any container provider runs the underlying infrastructure for you and you do not have SSH access to it, then Sandfly will not be able to connect and scan those containers. However, if those containers are running an ssh server, which is atypical but not unheard of, then Sandfly should be able to connect and scan the inside of the container that way.

Can I clean up inside /var/lib/docker/?

  • Sandfly does not provide its own method for cleaning up /var/lib/docker/ due to the risk of data deletion. However, Docker provides the command 'docker system prune' (official Docker documentation at: https://docs.docker.com/engine/reference/commandline/system_prune/) that can be used to remove unused data, including volumes with the added '--volumes' flag, that have accumulated on your Sandfly server and / or nodes.

  • ❗️

    IMPORTANT: Volume pruning can delete your database, so be very careful!

    If the sandfly-postgres docker container is NOT running, volume pruning will delete your database! Also, if non-Sandfly docker containers are or were on a target host, this process will affect every non-running container unless filters are applied.

  • Before using the command, we first recommend making a backup of the data. Next, open a shell with root privileges on either a Sandfly server or node host. If pruning the server, check that the container named sandfly-postgres is running:

  • docker ps

CONTAINER ID   IMAGE                           COMMAND                  CREATED       STATUS       PORTS                                                                            NAMES
4ff3b716aadf   quay.io/sandfly/sandfly:5.0.2   "/opt/sandfly/start_…"   10 days ago   Up 10 days   0.0.0.0:80->8000/tcp, :::80->8000/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp   sandfly-server
1cb5d910cbbc   postgres:14.9                   "docker-entrypoint.s…"   10 days ago   Up 10 days   5432/tcp                                                                         sandfly-postgres

  • Finally, run the prune command, which will remove all unused containers, networks, images (both dangling and unreferenced), and optionally, volumes:

  • docker system prune --all     # Optionally add '--volumes' to prune volumes

Operating Systems:

What commands will Sandfly use on the endpoints?

  • When Sandfly first logs into a host it uses several native Linux commands, which are run as the associated user, in order to examine the system in order to determine what that host is. Below is a list of those native commands, full paths are used where possible:
    • cat
    • chmod
    • echo
    • head
    • id (additionally ran via sudo)
    • ls
    • mkdir
    • pwd
    • rm
    • sh
    • tail
    • tr
    • uname
    • which
  • Outside of the native Linux commands, a Sandfly binary is pushed to a randomly named directory in the home directory of the associated user. Its default name is 'sandfly', however that binary name can be changed to one or more alternate names within the server's configuration. The directory will be in the format of <timestamp>.<random_chars> and the entire directory is removed after the scan finishes.

What version of Linux will Sandfly work against?

  • Sandfly is capable of scanning most Linux variants including embedded versions. It works on Linux versions running Intel, AMD, Arm or MIPS CPUs without any special modifications and only requires that your Linux host runs SSH. The application has been tested against the following distributions:
    • AlmaLinux
    • Amazon Linux Images
    • Arch
    • CentOS
    • CoreOS
    • Debian
    • Digital Ocean Linux Images
    • Fedora
    • Microsoft Azure Images
    • Oracle Linux
    • Raspberry Pi and other embedded systems
    • RedHat
    • Rocky Linux
    • Suse
    • Ubuntu
    • Customized Distributions

Can Sandfly scan Mac OS?

  • The Mac OS uses an unrelated kernel, which started life as the Mach kernel as a university research project building on an old BSD Unix kernel, and evolved into the XNU kernel for NeXT, which Apple eventually acquired and used as the basis for Mac OS X. Also the command line userspace in Mac OS is also from the BSD world, not the GNU userspace of Linux.
  • Which is all a long way of saying that no, Sandfly can not scan Mac OS, because despite the similarities of the command line interface, behind the scenes Mac OS is an entirely different beast whose kernel does not share even a single line of code with Linux. And the way Mac OS has evolved, it is even further from how the other non-Linux commercial Unixes do things in a lot of areas.

Sandflies:

What caused a "Sandfly halted with a SIGKILL signal" error?

  • SIGKILL is consistent with what the Linux OOM killer would use to terminate processes. Or there could be a malicious actor on the host.
  • If it was the Linux OOM killer, there should be an entry in the dmesg or kern logs. e.g. try dmesg | grep -i 'out of memory' and see if it mentions anything about killing a Sandfly associated process. Otherwise investigate that error as a potential security incident.

How do you change the timeout period of a Sandfly?

  • Complete the following steps to deactivate the original Sandfly and in its place use a customized version for future scheduled scans:

  • 📘

    NOTE: Regarding this solution...

    • Requires a license that allows for the creation of custom Sandflies.
    • Timeout values are in seconds and maximum timeout allowed is 3600 seconds (1 hour)
  1. Locate and open the sandfly details page for the sandfly that you would like to customize.
  2. In the upper right corner, click on the Deactivate button if the sandfly is active.
  3. In the upper right corner, click on the Clone button, which opens the custom sandfly editing form.
  4. Inside the box where the JSON is displayed look for "name" near the top and add some unique text at the end of the value to make the name of the sandfly unique and descriptive for your needs, for example "_10mTimeout".
  5. Inside the same JSON box, look for "active" and "custom" keys and change both of their values from false to true.
  6. Inside the same JSON box, look for "max_timeout" and change the value to the desired duration.
  7. Click on the Save button at the top or bottom of the page, which will create a new, custom Sandfly.
  8. Adjust any schedules, whitelists, and/or result profiles as needed to support the custom sandfly.

What happens to custom sandflies as we update and improve the product?

  • In the past we have auto-updated custom sandfly modules to new formats without customers needing to do anything. It is very likely we will continue this tradition for the foreseeable future.

Secure Shell (SSH):

Why is a scan returning a "SSHException: EOF during negotiation" error?

  • First, from a shell on a Sandfly Node host, try to manually connected via SSH to the host in question using the same credentials.
  • Next, check the sshd_config (usually located at: /etc/ssh/sshd_config) to confirm that the location (path and file name) for the "Subsystem sftp" entry exists and is correct on the host that is reporting this error. The line in the config should look something like this (depending on your OS):
    • Subsystem sftp /usr/lib/openssh/sftp-server
  • If it is not valid, add or correct it in the configuration file, reload sshd (e.g. service sshd reload or similar method) to use the updated value, and finally run a manual scan via the Sandfly UI for that system to see if it now can connect properly by Sandfly.
  • If that succeeds, apply the same process to any remaining hosts with this situation. If it does not work, please open a Sandfly support ticket.

Sensitive Data:

Is sensitive data encrypted in transport? (e.g. system-to-client)?

  • We encrypt all interactions from the UI to the server. We also encrypt all data to/from the systems being investigated and also that data sent back to the server.

Is sensitive data encrypted in storage (e.g. disk encryption, at-rest)?

  • We encrypt all credentials we store in the server at all times. However results, server configurations, etc. are not encrypted in the database.

Updated 2 months ago