Checking mirrors pings in Scala

June 11, 2012 Artem Software, 0

I’ve recently was installing CygWin on one of my machines and ran into question I run into all the time: which mirror to select to minimize time to load all my stuff. So I decided finally to write a small Scala program to ping mirrors and find out which one is the closest.

I decided to go with the most straight-forward, easiest way I could do. Firstly, I’ve extracted list of mirrors using my favorite Snagit and just pasted it to multiline read-only variable:

 Scala | 
 copy code |
?
 
1
  val urlListStr = """http://mirrors.163.com
2
http://box-soft.com
3
http://cygwin.petsads.us"""

Then what we have to do is obvious:

1. split;
2. extract server address;
3. ping and parse output (or check time);
4. sort by time and select 5 best;
5. print out best 5.

Split

is easy and straight-forward

 Scala | 
 copy code |
?
 
1
  val allUrls = urlListStr.trim().split("\\s+")

Extract server address

Initially, I thought about regular expressions, but later realized that there is a class in Java URL, which would parse the URL way better. Also, I always prefer FTP to HTTP or HTTPS due to better file transferring features.

 Scala | 
 copy code |
?
 
01
  val hostToUrl = HashMap[String, String]()
02
  allUrls.foreach(urlStr => {
03
    if (urlStr != None) {
04
      val hostName = new URL(urlStr).getHost();
05
      // FTP should be more efficient and robust
06
      if (urlStr.toLowerCase().startsWith("ftp") || !hostToUrl.contains(hostName)) {
07
        hostToUrl.put(hostName, urlStr)
08
      }
09
    }
10
  })
11
  val hostsList = hostToUrl.keySet

Actual ping and parse output

We’ll do it in two steps. Functionality first.

 Scala | 
 copy code |
?
 
01
  val pingMap: Map[String, Int] = new HashMap[String, Int]()
02
  val averageMsPattern = new Regex("""Average =\s+(\d+)ms""", "ms");
03
 
04
  for (host < - hostsList) {
05
    actor {
06
      val fullText = Seq("ping", host).lines_!.mkString(" ")
07
      val firstResult = averageMsPattern.findFirstMatchIn(fullText)
08
      if (firstResult != None) {
09
        val result = firstResult.get
10
        val pingMs = result.group("ms").toInt
11
        pingMap.put(host, pingMs)
12
      }
13
    }
14
  }

I loved the way how it is easy in Scala to run a command and get output back:

 Scala | 
 copy code |
?
 
1
val fullText = Seq("ping", host).lines_!.mkString(" ")

lines_! compare to lines will not throw an error in case if return code is not 0. This is exactly what we need since we check output ourselves anyway.

Pay attention to the way we parse output. It is Windows-specific at this point. I might extend it to support Linux/OS X a bit later.

Now, obviously it takes enormous time in just straight waiting. Let us make it concurrent to leverage ability to execute some while others are waiting:

 Scala | 
 copy code |
?
 
01
  val pingMap: ConcurrentMap[String, Int] = new ConcurrentHashMap[String, Int]().asScala
02
  val averageMsPattern = new Regex("""Average =\s+(\d+)ms""", "ms");
03
  val allFinishedLatch = new CountDownLatch(hostsList.size)
04
 
05
  for (host < - hostsList) {
06
    actor {
07
      val fullText = Seq("ping", host).lines_!.mkString(" ")
08
      val firstResult = averageMsPattern.findFirstMatchIn(fullText)
09
      if (firstResult != None) {
10
        val result = firstResult.get
11
        val pingMs = result.group("ms").toInt
12
        pingMap.put(host, pingMs)
13
      }
14
      allFinishedLatch.countDown()
15
    }
16
  }
17
 
18
  allFinishedLatch.await()

Here is the leverage of a JVM language: I can use java.util.concurrent package in full extent. With maps, latches, etc. Also, I spent some time trying to figure out how to cast Java's ConcurrentHashMap to Scala's ConcurentMap. Apparently you should

 Scala | 
 copy code |
?
 
1
import scala.collection.JavaConverters._

and use ".asScala" on the object, which looks pretty weird for a Java programmer. And pay attention how elegant Scala's actors look like! :)

Sort by time and select 5 best

This wouldn't be an issue with Java either with Collections.sort() method, but in Scala it is much shorter, because you tell it how to sort in place:

 Scala | 
 copy code |
?
 
1
 val best = pingMap.toList.sortBy(t => t._2).slice(0, 5)

Print out best 5

Here we use Scala way to iterate through a collection, although simple mkString() is not what we want:

 Scala | 
 copy code |
?
 
1
  best.foreach(t => println(t._2 + "\t" + hostToUrl.get(t._1).mkString))

Full source code

could be found here.

concurrent, Scala

What to do with InterruptedException in Java ArrayList is faster than LinkedList for add() operation