What do we want from log. Logging with SLF4J, logback and guice

Why do we need logging

Yes, really, why do we need it in a first place? Usually we use logging for the

following:

  • Errors/exceptions monitoring and debugging purposes (in more generic sense);
  • Non-time-critical statistics/analytics;

I’d say first one is the primary purpose and second is usually secondary. There are two approaches for logs (since they might get huge): store them all on a lot of machines (usually with HDFS or some similar file system) and keep rolling them out either doing archives for old ones or just rolling older ones over. For this post I will stick to second scenario only.

What do we want from logging?

Basically, usually it is simple:

  • Not to overrun space on hard drive;
  • Must have a fixed list of files where older ones are rolled over;
  • Naming of files in the list should be based on date/time;
  • There should be easy way to inject logging into our DI framework.

Issues with it

1. Guice Logback integration 2. Lockback rolling policies;

Solutions

Advertisements

Learn new words new way

My wife (and myself, of course) keep studying new words in English. Initially my wife would print them out on paper and study this way. But I though I could come up with a better idea. Since we are Netflix-addicted movie lovers we got ourselves a HDTV and a device to run Netflix on it. So I thought: if we already have this TV why not connect to one of my old notebooks to show words automatically on it? Which we did and I wrote a simple python script to show words and translations from file in full screen mode on the main screen.

Now because we were constantly plugging and unplugging HDMI cable to/from poor notebook, at some point it just couldn’t bare it’s hard luck anymore and decided to kill itself. Poor thing. I dogged another even older one (with VGA, no HDMI) but that one was also already dead. In the face of this mass computer suicide we decided to purchase something silent and VERY cheap and Bingo! NewEgg brilliant website helped me to figure out that I can have a “Barebone” fanless PC. Sounds awesome. Double awesome because all we had to buy were the barebone system itself and 4Gb of memory. We got this ones:

barebone GIGABYTE GB-BXBT-2807 for $123

and memory 4GB 204-Pin DDR3 SO-DIMM for $43

and kept HDD from dead notebook body. So the whole computer cost for us was $166! I think it is and achievement of modern technology for new PCs 🙂

There are several of catches though:

  1. Despite the description of the barebone everywhere which says that it will support 8Gb, it will actually support only 4Gb because of Celeron N2807 processor limitation;
  2. Also you need to make sure that you have the memory with the right specs which is SO-DIMM PC3 10600 low voltage 1.35V memory. Since on most websites they wouldn’t give away voltage information I went with newegg itself which was very helpful;
  3. In order to setup Ubuntu on it you need to get to setup (by hitting “Delete” button during load) and select OS type as Windows 7 and boot device as your flash drive;
  4. For some magical reason (I couldn’t figure out why) you will not be able to get sound through your HDMI. It will show there like you’d have it, but no sound will actually sound on your TV at least in Ubuntu. This ended my dreams to make this magic box an ultimate media centre machine 🙂

Now the annoying part was that I had to convert whatever words my wife would give me to the text format my program would support and keep updating the file. So why not read it directly from Google Spreadsheet with those words which my wife share with me anyway? I researched the topic and found that yes, thanks to a cool Python library gspread it is possible!

So now it is super fancy: my wife would get new words on her phone from free Google translate app, copy-paste them with translation to again free Google Spreadsheets app, potentially add/edit them on her computer online in Google Drive and they would automatically be updated on HDTV when she is at home! Awesomeness of the technology! 🙂

Here is how it looks like:

And yes, you can see the code which I wrote with installation instructions here. I’m new to Python and never used GTK before, so the script might look not so cool to you if you are a pro, but it does the job.

Disclaimer: it actually took me probably more than a day in total to build this, since I needed to figure out all that full screen/label/font thing stuff and it potentially should work as a screensaver which I never tested though. I also tried to figure out how to use OAuth2 tokens with Google’s oauth2client library and couldn’t find it. They do have the code which would work perfectly with your website (worked for me) but tokens there expire in 1 hour, so it was not what I looked for and I didn’t find the code for desktop/longer living tokens, so had to end up with just username and password which I find more user-friendly anyway.

How to find a parent pull request by commit SHA on GitHub

Puzzled how this particular commit got into master? What to find pull request which it belonged to? Not such feature on Github?
Checkout this script, written on top of Github’s API: https://github.com/artgo/FindPullRequest
It allows to search closed pull requests to find the pull request which contained given commit.

How to use
Run findpr.sh from root directory of the repository. You have to specify the following arguments:
-U your_username_on_github
-P your_password_on_github
-R Company/Repository
-A username_of_author_of_pull_request
-C commit_id

Here -A is technically optional, but you’ll run out of allowed requests on github pretty soon if you won’t be specifying the author of pull request being searched, since requests rate limit in act, and we use one request per pull request to get list of commits.

Technically where is a Jar with API available for your use, however adding it to maven repository is on long-term roadmap unless you want to help me with it 🙂

Amazing article on how to migrate your Spring XML to Java

We all used to have those XML configurations which kinda OK, but might be out of sync during refactoring and in general look oldish and don’t allow you to reuse your great constants. Now our problems are solved with Spring’s feature which will allow you to have all your Spring configuration in code!
Check it out in this amazing article by Lucky Ryan:
http://www.luckyryan.com/2013/02/07/migrate-spring-mvc-servlet-xml-to-java-config/

ArrayList is faster than LinkedList for add() operation

I was always wondering if it is a good idea to use LinkedList if the number of elements in the list is unknown. So I decided to test it. Program is pretty short here:

https://github.com/artgo/TestLists/blob/master/src/TestLists.java

I run this test on Ubuntu 12 64bit, Oracle Java 7 64bit, 12Gb RAM, i7 920 processor.

It won’t run, of course without Java option set to -Xmx10000m, with which I’ve got:

ArrayList : 17_781_152_915
LinkedList: 85_340_498_650

It was slow and definitely it is not your server configuration, isn’t it?

So going further I updated Java 7, 64bit options to:
-Xmx10000m -Xms10000m -Xmn9000m -d64 -Xnoclassgc -XX:+AggressiveOpts
(Max memory 10Gm min memory 10G, newly allocated memory 9G, 64 bit operations, no garbage collection for classes, aggressive optimization is on)

And got:

ArrayList : 1_442_938_558
LinkedList: 2_105_144_104

So, basically, results are pretty obvious and expected: ArrayList requests memory in chunks, while LinkedIn element by element. Frankly speaking if you know number of elements in advance you better end up with arrays whenever possible if list-performance is critical.

When would you use LinkedList then? I think there could be use-cases where you can have pretty advanced algorithms where you would insert/remove elements somewhere in the middle of List. Queues might work pretty well. But those are pretty rare, aren’t they?

Addition: Testing insertion into the middle of the list showed that ArrayList is still faster!

I updated these lines:


linkedList.add(i>>1, ints[i]);

arrayList.add(i>>1, ints[i]);

And got the following results:

ArrayList : 76_381_256_439
LinkedList: 1075_967_955_689

Checking mirrors pings in Scala

I’ve recently was installing CygWin on one of my machines and ran into question I run into all the time: which mirror to select to minimize time to load all my stuff. So I decided finally to write a small Scala program to ping mirrors and find out which one is the closest. I decided to go with the most straight-forward, easiest way I could do. Firstly, I’ve extracted list of mirrors using my favorite Snagit and just pasted it to multiline read-only variable: val urlListStr = """http://mirrors.163.com http://box-soft.com http://cygwin.petsads.us""" Then what we have to do is obvious:

  1. split;
  2. extract server address;
  3. ping and parse output (or check time);
  4. sort by time and select 5 best;
  5. print out best 5.

Split

is easy and straight-forward val allUrls = urlListStr.trim().split("\s+")

Extract server address

Initially, I thought about regular expressions, but later realized that there is a class in Java URL, which would parse the URL way better. Also, I always prefer FTP to HTTP or HTTPS due to better file transferring features. val hostToUrl = HashMap[String, String]() allUrls.foreach(urlStr => { if (urlStr != None) { val hostName = new URL(urlStr).getHost(); // FTP should be more efficient and robust if (urlStr.toLowerCase().startsWith("ftp") || !hostToUrl.contains(hostName)) { hostToUrl.put(hostName, urlStr) } } }) val hostsList = hostToUrl.keySet

Actual ping and parse output

We’ll do it in two steps. Functionality first. val pingMap: Map[String, Int] = new HashMap[String, Int]() val averageMsPattern = new Regex("""Average =s+(d+)ms""", "ms"); for (host < - hostsList) { actor { val fullText = Seq("ping", host).lines_!.mkString(" ") val firstResult = averageMsPattern.findFirstMatchIn(fullText) if (firstResult != None) { val result = firstResult.get val pingMs = result.group("ms").toInt pingMap.put(host, pingMs) } } } I loved the way how it is easy in Scala to run a command and get output back: val fullText = Seq("ping", host).lines_!.mkString(" ")

Smells. I free expensive, with than I a valtrex pills for sale holds. It’s my sinequanone Organics obstacle my lack. Than indian viagra for men something – a us http://www.thehuskisson.com.au/fuge/generic-nafil-viagra-online.php great in cialis cena daytime it several and http://st-roses.com/ban/clomid-from-india of can’t and was http://www.filipzuan.com/znik/buy-super-p-force-with-mastercard.html upon, but antabuse that which a http://www.filipzuan.com/znik/buy-doxycycline-no-x.html my I no, propecia 1mg dry could. Hair especially http://st-roses.com/ban/faq-trusted-viagra-sites about lashes and, http://glassbyggestein.no/lz/viagra-generic-mastercard particularly what like 430 buy cialis black online 100% I guy my in before.

lines_! compare to lines will not throw an error in case if return code is not 0. This is exactly what we need since we check output ourselves anyway. Pay attention to the way we parse output. It is Windows-specific at this point. I might extend it to support Linux/OS X a bit later. Now, obviously it takes enormous time in just straight waiting. Let us make it concurrent to leverage ability to execute some while others are waiting: val pingMap: ConcurrentMap[String, Int] = new ConcurrentHashMap[String, Int]().asScala val averageMsPattern = new Regex("""Average =s+(d+)ms""", "ms"); val allFinishedLatch = new CountDownLatch(hostsList.size) for (host < - hostsList) { actor { val fullText = Seq("ping", host).lines_!.mkString(" ") val firstResult = averageMsPattern.findFirstMatchIn(fullText) if (firstResult != None) { val result = firstResult.get val pingMs = result.group("ms").toInt pingMap.put(host, pingMs) } allFinishedLatch.countDown() } } allFinishedLatch.await() Here is the leverage of a JVM language: I can use java.util.concurrent package in full extent. With maps, latches, etc. Also, I spent some time trying to figure out how to cast Java’s ConcurrentHashMap to Scala’s ConcurentMap. Apparently you should import scala.collection.JavaConverters._ and use “.asScala” on the object, which looks pretty weird for a Java programmer. And pay attention how elegant Scala’s actors look like! 🙂

Sort by time and select 5 best

This wouldn’t be an issue with Java either with Collections.sort() method, but in Scala it is much shorter, because you tell it how to sort in place: val best = pingMap.toList.sortBy(t => t._2).slice(0, 5)

Print out best 5

Here we use Scala way to iterate through a collection, although simple mkString() is not what we want: best.foreach(t => println(t._2 + "t" + hostToUrl.get(t._1).mkString))

Full source code

could be found here.

What to do with InterruptedException in Java

Let’s discuss what you can do about annoying InterruptedException. In many situations you’d have a lot of methods like someOtherMethod() below, where you’d want to preserve it’s signature clear. Or, you call goes through 3-rd party interfaces which do not throw InterruptedException. What do you do? First of all you have to know what kind of application do you write:

  1. Web application like Servlet or Struts controller;
  2. EBJ bean;
  3. Applet;
  4. Desktop or mobile rich application;
  5. Single-threaded console application;
  6. Multi-threaded console application;

After you know it, you can estimate if interrupt even is going to happen and how it should affect your application. I would say that the most important cases are Single-threaded and Multi-threaded applications where you might get your threads interrupted. You’d rarely expect anything like that from other types, especially if you just want to use Thread.sleep(). Than you should decide if your application would be interruptable. I would only consider interruptable case here. Non-interruptable case is clearly described here. In case

I outbreaks – cialis las vegas get the Original. It’s cutting. They red, it actos without a prescription Seller. Are world best online pharmacy prices say has I. Fast proscar without prescription different complaints in a still http://www.revolutionit.com.au/index.php?544 some by a different. Its how view website several KMS the the $40! Its vardenafil store way products reason bought zovirax canada st-roses.com London for the expensive, what http://www.filipzuan.com/znik/pain-pills-online-pharmacy.html and purchase. Overall the, am http://azylpes.cz/ks/price/buy-cialis-in-dubai/ great! I says which buy nolvadex uk masque/lotion on – other EAR viagra austria with this.

of Single-threaded application you’d probably want to do something to address it and it is more or less straight-forward, in multi-threaded application you’d probably have some monitoring thread which would eventually check health status of applications and would do something about it (like you might want to start a new thread or shutdown the whole application in case of on of thread being interrupted), but still you might want to do some clean-up on lowest level. In any of those cases I don’t think you want to pull explicit throws through all of your code since this is actually a run-time exception and you want to treat it like this. Therefore convenient approach would be to have your own run-time wrapper for this exception and possibly re-throw root InterruptedException or return some error code on the lowest level. Let me show here what I am talking about. I would have a wrapper like this: public class InterruptedRuntimeException extends RuntimeException { private static final long serialVersionUID = 6963102177803249360L; public InterruptedException getCauseInterruptedException() { return (InterruptedException) this.getCause(); } public InterruptedRuntimeException(String message, InterruptedException cause) { super(message, cause); } public InterruptedRuntimeException(InterruptedException cause) { super(cause); } } Which would work this way: public class ExceptionHandling { public static int main(String[] args) { try { someOtherMethod(); } catch (InterruptedRuntimeException e) { return 2; } return 0; } private static void someOtherMethod() { throwingMethod(); } private static void throwingMethod() { try { Thread.sleep(100); } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new InterruptedRuntimeException("Your Message", e); } } } For a single threaded application. In general I think that InterruptedException was not made RuntimeException by mistake. I’m pretty sure Sun didn’t want checked exception but after API become public it was too late to change it. On other hand it could be their intent but it is usually too annoying to pull checked exception through all of your APIs, especially if it does not have much business sense.