Network Attack Resolved

Chinese attempt at brute force attack foiled.

On Saturday November 11th, 2018 at 04:30 EST SoniXCast was contacted by the United States Federal Bureau of Investigation that there was a brute force attack occurring on the SoniXCast edge network that serves US government systems. Within a short period of time the attack expanded to other SoniXCast networks in Canada and Europe that serve federal and commercial services including retail branch services.

The attack was mounted from the US.

The attack was mounted from 3 separate US location from Virtual Private Servers located in Atlanta Georgia, Dallas Texas and San Francisco California. Payment came from an offshore financial services company known to be associated with Chinese Intelligence Services. The attack was in form of a bot that would attempt multiple password variations in order to gain root access to a system. By evaluating TCP headers, technicians were able to backtrack connections to a server in Taiwan China.

The Resolution

SoniXCast emergency attack protocol was immediately implemented which confuses most modern network attacks. However, the protocol also confuses customer systems so some minimal downtime was experienced. There seems to have been a timeout associated with the attack script so that when the requested ip-address and port was no longer available, the bot gave up and moved on to another system which minimized downtime overall.

SoniXCast is cooperating with the US federal government and has contacted cyber attack units in countries where the attack on SoniXCast networks occured. Once a full report has been issued, the network team will evaluate and advise if further actions are necessary.

Which Url should I use? Relay vs Redirect vs Main

With the recent server ip-address changes, many customers have found out the hard way that 3rd party aggregators like TuneIn and others are just not aggressive enough when it comes to maintaining their domain name services (DNS). Serious aggregators such as iTunes, Roku and Sony have more robust network operations and correctly update when we do which is why customers using those platforms don’t have as many issues.

So, why change the IP-Address anyways?

Our network is hostname centric for reasons that will become clear as you read on. Those who been with us longer remember a time when we were being attacked on an almost daily basis causing service interruptions. Since then we’ve locked things down quite a bit and part of that action was to implement ‘Floating IP‘ and ‘AnyCast DNS‘ technology.

A Floating IP is an IP address that can be instantly moved from one System to another in the same datacenter. Part of a highly available infrastructure is being able to immediately point an IP address to redundant systems.

Anycast DNS is a network addressing and routing methodology, in which datagrams from a single sender are routed to the topologically nearest node in a group of potential receivers.

Our AnyCastIP technology implements a hybrid version of both technologies where an ip-address pool is predefined with DNS hostnames that are rotated at random intervals or when malicious activity is detected which are assigned to multiple redundant systems.

Hackers looking for new victims routinely scan whole ip networks looking for vulnerable systems. Streaming technology (regardless of vendor) requires the use of open network ports in order to communicate which introduces HUGE security holes. Our Intrusion Detection Technology watches for malicious connections and if something fishy is suspected, changes the ip-address to ward off any possible attacks.

This, of course, plays havoc with those still routed in the concepts of ‘static‘ ip-addresses as each hostname must be manually updated each and every time one or the other changes in order to achieve proper functionality. Which is why customers should NEVER use the IP-Address! Ever!

Enough Geek Speak! Which Url should I use?

We offer 3 separate Urls in order to cover the widest field of application. On the BoomBox Dashboard page, customers will see 3 ‘thumbs up’ icons colored according to application in the ‘My Station‘ section:

Relay Url (Green – Website and Players):

The relay url is what customers should put in 3rd party players, on their website and generally share with listeners. It is the most compatible, the most reliable and the url never changes as long as the account is active.

The Relay Url is funneled through a highly customized Icecast stream server cluster hosted on 24 geographically dispersed network nodes) that normalizes and buffers the stream for fastest possible startup and rock solid, high quality, playback. And since Icecast (more so by far than ShoutCast) works with every known player out there, it is compatible with just about anything including computers, tablets, set-top boxes and cellphones.

Redirect Url (Blue – 3rd party Software and Aggregators)

Aggregation Services (i.e. TuneIn), Broadcasting software (i.e. SAM Broadcaster) and 3rd party tools (i.e. RadioToolBox) need to query ShoutCast’s proprietary ICY protocol in order to function correctly. The redirect url merely forwards port requests directly from our worldwide nodes to the main url, but more importantly the url never changes as long as the account is active.

Main Url (Red – Backup only)

Those having difficulties broadcasting live using 3rd party broadcasting software can use the main url as a fallback, but be advised that the url can change at any time.

Synopsis DDNS issues SoniXCast.com

Like many other hosting providers, SoniXCast relies on 3rd party dynamic DNS (domain name system) to ensure that our website and applications are available worldwide.

Between 06/28/2017 and 07/01/2017 our previous DDNS provider (BuddyDNS) experienced catastrophic network issues and they have, to date, not resumed service. We have therefore retained the services of 2 other 3rd party providers (Amazon.com, Dynu.com) to provide DDNS services.

DNS is the mapping of domain names (i.e. sonixcast.com) to an internet protocol (IP) address which enables browsers and other applications to find SoniXCast services. Dynamic DNS a.k.a AnyCast DNS is a modern version of DNS which also helps route requests to the server more expediently.

Like yourselves and your listeners, SoniXCast relies heavily on DDNS internally to support our monitoring and load balanced systems. Whenever there is an issue with DDNS, then those systems can get out of sync and problems occur like missing statistics, inability to reach the control panel or website from some locations, problems uploading files, services restarting without apparent reason and so forth…

In addition, over the last 3 months, BuddyDNS has notified our network support team that our account had gone over quota (50 million hits a day) half way through the month and threatened with suspension. Our network team was evaluating other providers when the BuddyDNS network went down.

On average sonixcast.com and sonixfm.com receive 42.3 million hits per day according to our internal counters. The majority of which come from white-label partners Sony (12 million), Apple (4 million), Microsoft (8 million) and Samsung (6 million). The rest come from retail accounts, diverse 3rd party aggregators and internal services.

We are confident that the new DDNS providers have adequate load capacity to serve SoniXCast well into the future and we wish we would have reacted sooner. However, it is nearly impossible to predict growth and popularity of a cutting edge system where no precedence exists and the decision was made to err on the side of caution.

For that we apologize to all our customers and partners and thank you for your patience and understanding.

SoniXCast Web Services Uptime Monitoring by UptimeRobot

SoniXCast has partnered with UptimeRobot.com to provide real-time monitoring solution for all SoniXCast services. Useful for diagnosing connection problems, consumers can view the current status of all SoniXCast services including the BoomBox control panel, Api, CDN, Nameservers, Servers, AnyCastIP and corporate websites.

We are offering customers two status pages where they can check on SoniXCast Web Services:

https://www.sonixcast.com/uptime: Is our own homegrown solution which is built into BoomBox and all other Web Services and Sites and display’s verbose uptime graphs and status’ for each SoniXCast Web Service.

http://status.sonixcast.com/: Provided by UptimeRobot has been setup in case of the unlikely event that the SoniXCast CDN (provided by Amazon S3) goes down. Users will need to enter “SoniXCastStatus16” in order to login and view current status.

Radionomy Flounders, Sonixcast Grows

Radionomy is currently suffering extreme network issues according to their facebook page.

Ever wonder why Sonixcast customers never experience such problems? In a word AnyCastIP. Never heard of it huh. Well AnyCastIP is Sonixcast’s patented digital content distribution network which serves as a backbone for all Sonixcast services and works diligently behind the scenes ensuring that broadcasts have the same high quality regardless of where in the world one listens in from. Continue reading “Radionomy Flounders, Sonixcast Grows”