Graphite: Help me interpret the whisper-info.py ou

Question

I have been trying to make sense out of the whisper-info.py output and it seems a little cryptic to me.

Currently this is my setup:

storage-schema.conf

[carbon]
pattern = ^carbon\.
retentions = 60:90d

[stats]
pattern = ^stats.*
retentions = 30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d

[everything_else]
pattern = .*
retentions = 30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d

storage-aggregation.conf

[min]
pattern = \.lower$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = \.upper(_\d+)?$
xFilesFactor = 0.1
aggregationMethod = max

[sum]
pattern = \.sum$
xFilesFactor = 0
aggregationMethod = sum

[count]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum

[count_legacy]
pattern = ^stats_counts.*
xFilesFactor = 0
aggregationMethod = sum

[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average

I have statsd in front of it, pretty minimal localConfig.js

{
graphitePort: 2003,
graphiteHost: "127.0.0.1",
port: 8125,
flushInterval: 30000,
}

now I send a packet to stasd to a completely new metric

echo "alex.foo:1|c" | nc -w1 -u 127.0.0.1 8125

and this is the output of the whisper-info.py:

# whisper-info.py /opt/graphite/storage/whisper/stats/alex/foo.wsp 
maxRetention: 604800
xFilesFactor: 0.300000011921
aggregationMethod: average
fileSize: 120988

Archive 0
retention: 604800
secondsPerPoint: 60
points: 10080
size: 120960
offset: 28

Why is there only one archive when I clearly have so much more retentions setup? are my schema rules not correct? what is the meaning of the output? is size the file size? offset?

Thank you for taking the time to answer!

Answer 1

I have found out that problem was with the syntax of my storage-schema.conf file. I found this handy tool validate-storage-schemas.py (found in /opt/graphite/bin/) and the output was pretty clear.

./validate-storage-schemas.py 
Loading storage-schemas configuration from default location at: '/opt/graphite/conf/storage-schemas.conf'
Section 'carbon':
  OK
Section 'stats':
  - Error: Section 'stats' contains an invalid retention definition ('30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d')
    Lower precision archives must cover larger time intervals than higher precision archives (archive4: 157680000 seconds, archive5: 157680000 seconds)
  OK
Section 'everything_else':
  - Error: Section 'everything_else' contains an invalid retention definition ('30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d')
    Lower precision archives must cover larger time intervals than higher precision archives (archive4: 157680000 seconds, archive5: 157680000 seconds)
  OK

now if I do a whisper-info on the files they make sense:

root@graftwo:/opt/graphite# whisper-info.py /opt/graphite/storage/whisper/local/test/diceroll.wsp maxRetention: 172800000 xFilesFactor: 0.300000011921 aggregationMethod: average fileSize: 2113528

Archive 0
retention: 21600
secondsPerPoint: 30
points: 720
size: 8640
offset: 88

Archive 1
retention: 2592000
secondsPerPoint: 60
points: 43200
size: 518400
offset: 8728

Archive 2
retention: 31104000
secondsPerPoint: 600
points: 51840
size: 622080
offset: 527128

Archive 3
retention: 62208000
secondsPerPoint: 1800
points: 34560
size: 414720
offset: 1149208

Archive 4
retention: 157680000
secondsPerPoint: 3600
points: 43800
size: 525600
offset: 1563928

Archive 5
retention: 172800000
secondsPerPoint: 86400
points: 2000
size: 24000
offset: 2089528

I am still wondering two things: - why didn't carbon-cache failed with incorrect syntax? - where are this hard defaults coming from? - what is this offset thingie?

Thanks! hope this helps someone else