I have been trying to make sense out of the whisper-info.py output and it seems a little cryptic to me.
Currently this is my setup:
storage-schema.conf
[carbon]
pattern = ^carbon\.
retentions = 60:90d
[stats]
pattern = ^stats.*
retentions = 30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d
[everything_else]
pattern = .*
retentions = 30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d
storage-aggregation.conf
[min]
pattern = \.lower$
xFilesFactor = 0.1
aggregationMethod = min
[max]
pattern = \.upper(_\d+)?$
xFilesFactor = 0.1
aggregationMethod = max
[sum]
pattern = \.sum$
xFilesFactor = 0
aggregationMethod = sum
[count]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum
[count_legacy]
pattern = ^stats_counts.*
xFilesFactor = 0
aggregationMethod = sum
[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average
I have statsd in front of it, pretty minimal
localConfig.js
{
graphitePort: 2003,
graphiteHost: "127.0.0.1",
port: 8125,
flushInterval: 30000,
}
now I send a packet to stasd to a completely new metric
echo "alex.foo:1|c" | nc -w1 -u 127.0.0.1 8125
and this is the output of the whisper-info.py:
# whisper-info.py /opt/graphite/storage/whisper/stats/alex/foo.wsp
maxRetention: 604800
xFilesFactor: 0.300000011921
aggregationMethod: average
fileSize: 120988
Archive 0
retention: 604800
secondsPerPoint: 60
points: 10080
size: 120960
offset: 28
Why is there only one archive when I clearly have so much more retentions setup? are my schema rules not correct? what is the meaning of the output? is size the file size? offset?
Thank you for taking the time to answer!
I have found out that problem was with the syntax of my storage-schema.conf file. I found this handy tool validate-storage-schemas.py (found in /opt/graphite/bin/) and the output was pretty clear.
./validate-storage-schemas.py
Loading storage-schemas configuration from default location at: '/opt/graphite/conf/storage-schemas.conf'
Section 'carbon':
OK
Section 'stats':
- Error: Section 'stats' contains an invalid retention definition ('30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d')
Lower precision archives must cover larger time intervals than higher precision archives (archive4: 157680000 seconds, archive5: 157680000 seconds)
OK
Section 'everything_else':
- Error: Section 'everything_else' contains an invalid retention definition ('30s:6h,1min:30d,10min:360d,30min:720d,1h:1825d,1d:1825d')
Lower precision archives must cover larger time intervals than higher precision archives (archive4: 157680000 seconds, archive5: 157680000 seconds)
OK
now if I do a whisper-info on the files they make sense:
root@graftwo:/opt/graphite# whisper-info.py /opt/graphite/storage/whisper/local/test/diceroll.wsp
maxRetention: 172800000
xFilesFactor: 0.300000011921
aggregationMethod: average
fileSize: 2113528
Archive 0
retention: 21600
secondsPerPoint: 30
points: 720
size: 8640
offset: 88
Archive 1
retention: 2592000
secondsPerPoint: 60
points: 43200
size: 518400
offset: 8728
Archive 2
retention: 31104000
secondsPerPoint: 600
points: 51840
size: 622080
offset: 527128
Archive 3
retention: 62208000
secondsPerPoint: 1800
points: 34560
size: 414720
offset: 1149208
Archive 4
retention: 157680000
secondsPerPoint: 3600
points: 43800
size: 525600
offset: 1563928
Archive 5
retention: 172800000
secondsPerPoint: 86400
points: 2000
size: 24000
offset: 2089528
I am still wondering two things:
- why didn't carbon-cache failed with incorrect syntax?
- where are this hard defaults coming from?
- what is this offset thingie?
Thanks! hope this helps someone else