The recent sneak peek publication of the 2017 “Verizon Data Breach Digest” highlighted how hackers had used a University’s own vending machines and other internet-enabled devices to hack their network and almost brought the campus to a halt.
You can read the full story, but in summary, the initial alert came from students complaining about slow network connectivity. Reacting as many of us would, the Help Desk team was not too concerned when the first couple of complaints came in (“We are sorry your Play Station is running slow…maybe you should get back to studying.”).
However, as the situation evolved and it became more apparent that there was something larger going on, the senior IT security team was notified. Further investigation by this team discovered that the domain name servers (DNS; which are responsible for doing network lookups) were producing an abnormal number of network messages related to seafood. The team’s firewall analysis identified that over 5,000 different systems, all part of the University’s Internet of Things (IoT) infrastructure, were making hundreds of DNS lookups every 15 minutes.
To reduce the resources necessary to manage systems on campus, everything from light bulbs to vending machines had been connected to the same network and were all configured to use the DNS servers. Malicious software had spread from device to device through the use of brute force attacks on devices that had default or weak passwords. Once the password was discovered, the malware was able to take full control of the device and change the device’s password, thereby locking the University out of the infected systems. It was this controlled group of devices, referred to as a botnet, which was generating an excessive number of messages in an effort to deliberately overwhelm the network.
After a moment of panic, in which the University thought they may have to replace all of the impacted devices or their entire network might be taken down, they identified a solution. The IT team was able to utilize an application to intercept the malware’s cleartext password and then login, update the password, and remove the infection.
There are a number of key takeaways from this incident:
- Know what is connected to your network. Do you have a comprehensive inventory of all devices (i.e. vending machines, lights, cameras, heating systems, etc.) that are connected to the Internet via your organizational network? Do you have a formal procedure in place to ensure that all devices are reviewed and approved before a connection is made?
- Secure any approved devices before connecting them. This was the first mistake that was made in this University’s case. If these devices had been secured with strong passwords, they wouldn’t have been such an easy target. It is also important to ensure that all connected devices are receiving regular updates as new security risks and threats emerge, and patches are released.
- Verify segmentation. According to the case study, these devices were supposed to have been isolated from the rest of the network; clearly, they weren’t. Be sure to create separate network segments for any internet-connected device and verify that appropriate firewalls are in place.
- Regularly monitor network traffic. Spikes in certain types of network traffic often indicate an issue that should be investigated immediately. If the University had been regularly monitoring traffic, they would have been alerted to the abnormal number of DNS requests prior to the student’s complaints and network connectivity issues.
- Regularly review logs. Log monitoring most likely would have revealed this issue sooner and allowed the University to get in front of the intrusion.
- Perform regular penetration tests. Penetration testing may reveal devices that you weren’t aware were even connected to your network and, even worse, if they are still configured with the default password.
Has your campus deployed many internet-connected devices? Manufacturers often focus first on functionality and fancy features, and security controls are included as an afterthought. Therefore it is important that you take the time to analyze all devices prior to implementing them in your environment.
In our previous article on the Internet of Things, we discussed how often botnets are built out of these devices and used to launch dedicated denial of service (DDoS) attacks; sadly this is precisely what happened to this university. We strongly recommend that any Internet-enabled device is assessed the same way you would a PC or workstation. Namely, you should verify that all devices are approved and secured before connecting them to any network resources, and follow industry best practices for protecting and updating the devices on an ongoing basis.
Some additional guidance from our RedLens InfoSec team:
[Wheeler]: When performing penetration tests, it is always exciting when we encounter IoT devices. Many have inherent security issues, and many lack the availability of regular updates. It can be very eye opening to see how one simple device can lead to a larger network compromise. Using this university as an example, imagine this: The devices on the IoT network were compromised and malware installed, but instead of just causing lots of network traffic, attackers could leverage the access to the DNS servers to replace DNS entries. When systems within their CDE rely on these DNS servers to resolve the IP address of www.ourpaymentprocessorgateway.com, those CDE systems receive the attacker’s IP address and send payment information directly to this attacker. By creating a man-in-the-middle situation, these insecure IoT devices leverage a supporting system, to affect the CDE. Although systems may not be within PCI-DSS scope, the overall security of the network should be considered.
Please contact us if you have any questions or would like to discuss a more comprehensive assessment of your organization’s network security infrastructure.
