BitNinja is a disruptive technology so there are some concepts that are important to understand in order to comprehend the way BitNinja works.
IP reputation is a very effective way of securing your server. It’s a database with information about various IPs in the world. BitNinja clients use IP reputation information automatically on their servers to make security decisions and to find out more about an IP address.
Every server with BitNinja can detect and defend a wide range of attacks. The server can send gathered incident information to our central database. Based on the type, timing, and amount of incidents an IP has in the database, it is categorized into one of the following lists:
If there is no information about an IP address, or based on the latest behavior the IP is not listed.
In traditional IP reputation terminology, we differentiate black and white lists. An IP can be trusted (whitelisted) or absolutely denied (blacklisted). This concept is very inflexible and this is the cause of the bad reputation that IP reputation lists have. If an IP is false-positively blacklisted, users will get angry at you because they can’t access the system they want to and they will be frustrated as they are unable to do anything about it.
That’s how the concept of greylisting was born. We wanted to represent a list of IPs we think may be malicious but we are not completely sure of it yet.
The greylist contains suspicious IPs that the BitNinja client handles with special care. BitNinja has different CAPTCHA modules for different protocols. The duty of a CAPTCHA module is as follows:
- Decide if the user is human or not
- Inform the user about the fact that his/her IP has been greylisted
- Provide a safe way for the user to delist his/her IP
- Save any requests made by non-human parties, growing the knowledgebase about the IP and the sin list.
- Honeypotting by pretending to be a vulnerable system so bots will try to connect
Traffic from an IP can be malicious and innocent at the same time. Think of an infected computer. The requests the virus will make are malicious but the requests the user makes by using his/her browser are innocent. If we completely block the traffic, the user will not know about the infection. Greylisting solves this by informing the user of the greylisting and helps the innocent users resolve their IPs.
If there are suspicious incidents about an IP address, the IP can be greylisted by some users. If an IP is user-greylisted, it means it is only greylisted by some users, not all BitNinja users. When we have enough information about an IP that is sending malicious requests, we move it to the global greylist. If an IP is globally greylisted, it is greylisted by all BitNinja servers.
You can check if an IP is listed by BitNinja using the web front end https://admin.bitninja.io or using our CLI front end that is called bitninjacli. You can check, add, and delete IPs from the greylist using both tools.
If one of your servers catches an attack and the IP is not yet greylisted, the IP will be user greylisted and this information will be distributed across all of your servers instantly.
If there is enough evidence that an IP is suspicious, we move it to the global greylist and distribute this information to every BitNinja protected server.
When an IP is globally greylisted and is still sending malicious requests, we identify it as dangerous. Such IPs are moved to the global blacklist we maintain. BitNinja clients will drop packets for IPs on the global blacklist. The false-positive rate of the global blacklist is very low, as there are many steps before we decide to blacklist an IP. Blacklisted IPs are moved back to the greylist from time-to-time to check if the traffic is still malicious or the system has been disinfected.
User level black/white lists¶
You can use BitNinja to maintain blacklists and whitelists across your servers. The BitNinja Dashboard or our command-line interface, BitNinja CLI, can be used to add/remove/check IPs against a user’s blacklist and whitelist. If you blacklist an IP, all of your servers will DROP any packets from that IP. You can add CIDRs to your blacklist or your whitelist via BitNinja Dashboard. When you whitelist an IP it guarantees BitNinja won’t interfere with that IP.
Whitelisting an IP will only bypass BitNinja iptables chains. There can be other iptables rules blocking the traffic. Whitelisting is not equal to universal ACCEPT, only to bypass BitNinja.
Whitelisted addresses are processed prior to blacklisted ones starting from BitNinja version 1.16.16.
Essential list provides protection against the most dangerous IPs. These IPs are often used by the most agressive hackers all around the world. When an IP generates more than 5000 malicious requests, BitNinja places it on this list. The essential list is part of our basic IP reputation package, so it’s available for every BitNinja user.
You can add whole countries to your blacklist or whitelist. The country blocking (or whitelisting) feature of BitNinja uses publicly available zone files to add the selected countries’ IP ranges to the user blacklist or user whitelist. The zone files can be found at http://www.ipdeny.com/ipblocks.
You can add countries on the Dashboard in the IP manager menu.
If you have ipsetv6 or ipsetv4 and add a country or remove it to the blacklist or whitelist, the ipsets will be reloaded on your server(s).
If you don’t have ipset, BitNinja uses iptables rules and simulated ipset. When you add or remove a country, the iptables rules will be reloaded.
For now, if you have more than one server and add a country to your blacklist (or whitelist), the selected IP ranges will be blacklisted (or whitelisted) on all your servers.
The country blocking feature is supported from BitNinja version v1.15.0
Searching in the BitNinja IP reputation database¶
You can search the IP history using the BitNinja Dashboard at https://admin.bitninja.io.
The search field is in the upper right corner of the UI. You can type an IP and choose the IP option to search in the BitNinja IP reputation database.
Railgun server configuration¶
Many of our customers who use Cloudflare are not able to see the proper visitor IPs in the logs, only the railgun server IPs. This is due to the fact that the requests are not directly coming from Cloudflare so mod_cloudflare will not restore the IPs of the visitor. That is why we would like to ask you to configure mod_cloudflare the following way:
Open your Apache configuration ( if you do not know where to find it, ask your hosting provider for assitance )
At the very end, add:
If you have more servers to add, it should look like this:
CloudFlareRemoteIPTrustedProxy xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx
Following this configuration, you will be able to see the adequate logs on our Dashboard.
If you need any further assistance, do not hesitate to contact our support team: firstname.lastname@example.org
It’s possible that BitNinja identifies internet crawlers as malicious bots due to their configuration and may greylist them due to security reasons.
To prevent this issue, we would like to ask those who have spiders, crawlers, or proxies to contact our team at email@example.com and provide their reverse DNS or if it’s not possible, then the fixed IP list. Our Technician Team will thoroughly analyze the activity of the bot and if they see that the questioned robot indeed performs only valid traffic, they will whitelist it upon concordance.
For the time being BitNinja does not support IPv6. If you are not using IPv6 it’s recommended to disable it. If BitNinja caught an IPv4 address, the attacker can still attack through IPv6. To disable IPv6 edit the file - /etc/sysctl.conf
$ sudo gedit /etc/sysctl.conf
And fill in the following lines at the end of that file
# IPv6 disabled net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1
Save the file and close it. Restart sysctl with
$ sudo sysctl -p
BitNinja will support IPv6 in the future.