Recent Changes - Search:
Site /

WhyJSON

Why JSON?

  1. JSON is JavaScript Object Notation = a hierarchy of name plus value paired data. 
    {
    "string_name":"my string",
    "integer_name": 123456,
    "double_name": 9876.54321
}
  2. Since the design of nmon data format, computers have come a long way and able to cope with much higher volumes of data, lots more stats, ability to save the data over a network, online graphing etc.
  3. It is an industry standard file format - nmon format is tricky in places to understand
  4. It is self-documenting in grouping stats together under a name and the names help too
  5. If more data is added to the output, tools can automatically work out how to store it
  6. Many modern performance tools and graphics tools accept JSON format
  7. JSON format is practically identical to the Python language "dictionary" native data structure making is a good choice for handling JSON data
    • This means a JSON file can be read and parsed extremely quickly and then the data you need to be extracted in simple Python statements.

Why the change to JSON format output than sticking with the older nmon format?

While writing nmonchart and other tools, I realised the nmon format is tricky to deal with. The format was dealing at the time with the need for extreme efficient data saving (reduced CPU time) and to keep the files small (spreadsheet limitations).

nmon was designed more than 20 years ago.

  • CPUs were 16 MHz but are now 4000 MHz (plus nearly 200 in a server and now with running SMT=8) = roughly 200,000 times faster
  • Memory was small like 32 MB was large - now we have 32 TB is large = 1,000,000 times larger
  • Disks were 200 MB now we have multiple PB in a 2U unit = 500,000 times larger
  • Networks 10 Mb/s and now 40 Gb/s = 4000 time faster

We now can spend more CPU time on gathering a lot more performance stats and stay below the noise level of CPU cycles plus we can store the stats over the network rather than local disks. JSON format allows:

  • a better description of the stats
  • allows for changes in resources like more or fewer disks and networks
  • grouping of stats that can make it clearer
  • Also the files created by njmon compress very well = a factor of 36 i.e. 36 MBs of data gzip's to 1 MB of file

njmon does limited data changes:

  • reports what the OS supplies to the reduced chance of screwing up
  • it reports a lot more stats than nmon - the user can decide it they are important or not
  • it reports incrementing counters as you would expect as data rates per second for the period between snapshots - i.e. not dependent on the period
  • the stats names are exactly as found on the operating system - this allows users to read OS documentation for the details of what the stats mean