I would like to choose a random seed for numpy.random and save it to a variable. I can set the seed using numpy.random.seed(seed=None) but how do you get numpy to choose a random seed and tell you what it is?
Number seems to use /dev/urandom on linux by default.
The full state of the MT19937 PRNG that underlies RandomState cannot be contained in a single (normally-sized, e.g. 32-bit or 64-bit) integer. It has an array of 624 32-bit integers for its state. Seeding with an integer actually runs a smaller, simpler PRNG to generate those 624 words. It is just a convenient way for humans to manually set the state of the PRNG to a state that can be consistently replicated. But most states that the PRNG gets into cannot be reduced back to a convenient 32-bit integer. That initializer PRNG cannot work "backwards" in this way. Instead, the whole state of RandomState is contained in that 624-entry array. You can get this array and set it using the get_state() and set_state() methods.
>>> import numpy as np
>>> prng = np.random.RandomState()
>>> state = prng.get_state()
>>> state
('MT19937',
array([2310623686, 364919541, 1436109096, 1457837701, 2852017530, 562204638, 1207376362, 2290452263, 250624867, 1687514807, 3242300311, 68301227,
497650124, 3782308076, 4180165271, 3190969185, 1284472452, 2868357773, 1148940887, 433865334, 643839653, 3091921054, 2157305915, 4079505239,
1396964105, 221256094, 2789328727, 3216471912, 1782932723, 1704818545, 3880597634, 2060476197, 2599008138, 1389874875, 56765165, 1173841349,
278528026, 714062321, 3587382791, 840507318, 2086996355, 3416087866, 3081938567, 946222923, 4259369972, 868558506, 2060774692, 3239317074,
4078800142, 3833877854, 1503749328, 3821805560, 1447854235, 995535877, 3762179650, 185008825, 149218213, 3469766149, 803379340, 3971043961,
3421104633, 2287066419, 2465098532, 4088166586, 2105722956, 1451099732, 3115885598, 4240224392, 3778829453, 4059831750, 2919989511, 4092928731,
922778621, 1805422791, 3344418665, 1738799711, 1367565729, 34977430, 4008589298, 2239856842, 1717530303, 32049105, 3468621644, 2269299060,
1664083607, 3996022881, 377407365, 4070209212, 4216115381, 2124999225, 1920630572, 2011423407, 1367187092, 4158622494, 487432561, 3536187733,
931951977, 749985693, 2812437433, 3902171864, 767004922, 3807520852, 796884475, 2794577773, 1481140267, 2247603372, 1053872430, 211335743,
2997489007, 4140013480, 1601875594, 1927437737, 3349007801, 2868575676, 3474179396, 595650352, 517981041, 1947095736, 170970294, 3253183597,
2873789192, 3386930182, 2047755893, 254974719, 2747566023, 4182212825, 1934990158, 1282861435, 404005052, 3237256048, 1737335951, 386655885,
640537519, 60176882, 1825713593, 86537970, 252007523, 3674897989, 3645447766, 972417578, 1860821974, 2688102651, 2481103756, 3672142036,
2961031222, 1709451377, 134371222, 4217784577, 3792528752, 1278543741, 291978547, 1987232116, 2685749450, 948431490, 3550698848, 1384058130,
302186886, 2966159795, 1981959565, 2602891721, 1814325871, 4148300386, 1211156469, 2945951607, 4132724234, 1221821676, 3057395063, 1563869020,
3762934166, 3303914085, 1910775932, 2241726842, 3836262483, 905479357, 2974032168, 3187395363, 3071243546, 3571439927, 3756380578, 53494506,
495375628, 2149633842, 1549467921, 403773184, 3774309942, 1767528278, 421982610, 579688614, 3735062896, 2128447283, 2545877077, 3013437905,
4067651631, 26043227, 3189924699, 1882256309, 431961449, 3637287121, 1409924095, 3834921204, 3796550515, 338734970, 1632375419, 3788135288,
153287562, 2302436235, 3852961194, 2073555800, 3034065218, 1997718747, 3343015031, 3198064720, 4286393046, 3338997777, 1383744819, 1553624825,
1183357509, 1141531260, 25823987, 2951322047, 4066666075, 3687780778, 3680053857, 478734258, 3674686218, 1457141125, 3673486342, 3224971043,
2786082270, 2282591016, 1210618789, 3735610308, 587294285, 4231880327, 3702701983, 13470000, 90747549, 876795924, 1489448380, 585176585,
2398768918, 3069244786, 2901497718, 4004899727, 1992450245, 1127097566, 713011674, 2083831719, 2923291311, 315998911, 1511233310, 1515243002,
621858088, 2398475656, 3029652473, 1011396654, 1854317252, 2735915680, 1489448619, 3836317799, 1678027486, 2429831383, 170989290, 651235170,
1457126476, 3694269669, 4248613755, 3161380741, 3396304589, 26218095, 4262314194, 3090365505, 2603976562, 1742639443, 3357356842, 2527908520,
2744118109, 764708873, 608716002, 218517036, 2028062957, 123264851, 3930797933, 1358280349, 3770182726, 1475205800, 4083653367, 728440387,
578359463, 3792859449, 2660424205, 866268419, 2680711984, 1892477918, 3473675890, 5948212, 590585309, 1434154869, 4019090587, 3447601971,
3777365598, 502271900, 933280098, 551410763, 4178545332, 2426657681, 435161245, 103552671, 2751130089, 1664159723, 2124278140, 3518289293,
1397473574, 4032873848, 3104766011, 3780526375, 146118438, 3497842141, 2078614647, 1431064844, 825222639, 954382890, 3170571595, 1418867403,
4133763948, 2773874577, 459104952, 3336058631, 791669682, 79496438, 1268256964, 1327605157, 3196785479, 3094404795, 3971934915, 967528556,
1680157581, 1508139540, 3821158380, 3603819236, 593155253, 1875654417, 3734837198, 3315972391, 2450938455, 1863178045, 619766009, 1376779265,
843230528, 1818810226, 1508689309, 1353144904, 3459699509, 734863896, 1593154156, 4178196553, 559982910, 1937392142, 3328058492, 2417976146,
3197182411, 2233439700, 196920494, 3714701774, 4104568606, 850977604, 382851029, 4143478133, 3024891142, 2455897904, 28681198, 3438784382,
578301023, 2215641381, 59642080, 2913625733, 2063824530, 2113835214, 563503294, 2261300428, 1156324177, 3080988993, 1485826140, 291045970,
3740234437, 2802003429, 804278225, 1715783317, 3683156408, 2855890524, 2390104305, 172369852, 3358371994, 1184782876, 2087670358, 840924195,
2727925375, 1806621317, 2785628046, 4163132724, 3580142689, 1107366902, 809125531, 3131770778, 1922818283, 888842000, 2875999147, 2752567229,
170460348, 1952532683, 1705378473, 1784443344, 1111435234, 2373828316, 1440965774, 3986117425, 849160375, 1233392480, 4073490673, 3948548975,
2317742686, 459747729, 3981827733, 97170450, 1906613346, 2296986726, 3107045483, 3301310854, 2005065797, 1047441812, 1340913878, 1305190832,
3414530672, 2739562683, 670592573, 3517927973, 3902124497, 4085960935, 823980090, 982263838, 1807290575, 1182843877, 3543714667, 1403590968,
329717243, 1055811172, 3550329386, 3998515559, 3251582755, 2201054306, 3347834116, 1211790680, 62972368, 88227180, 2967020240, 1937245345,
524567284, 2915223835, 1039263578, 931149438, 2102426452, 4178383760, 2534760455, 3961494901, 359726861, 2377704223, 3980574430, 3941075859,
3025460765, 1087397787, 1520908724, 3979084899, 3800423495, 139799221, 644687977, 1080267251, 599331265, 379370383, 3716980301, 2450151406,
1223752702, 300351842, 295249068, 1870733374, 2986315084, 1323736886, 306347366, 2697516131, 3896227616, 2556699990, 578928278, 2356101730,
171880210, 722319049, 740054230, 3855145369, 1468149367, 311954206, 4099077708, 2941657479, 119786529, 3197372768, 2115311247, 2469241538,
2636086203, 2206369175, 374899905, 3730393440, 2288141890, 719446033, 4096038147, 4294410470, 19272682, 1964868281, 3192582061, 3934009074,
1135732985, 682697379, 3290113635, 1489105351, 347638343, 147496092, 4175447059, 341595821, 3117140389, 1003085251, 1889252416, 913732530,
3459561042, 3662473182, 3839509269, 1519115576, 677113, 597583022, 3031451769, 607339281, 55523370, 2676982537, 1238056185, 1550912054,
3112284354, 1345961520, 1541909925, 3726796822, 2696250478, 3254836471, 1362613883, 3129122359, 1550126204, 129690651, 2386622242, 407302605,
1753882614, 2376840660, 1076064874, 2449053256, 3162294193, 3779999195, 3925427556, 2601606505, 1901788890, 2217639773, 406665902, 3640687773,
2061876750, 968895635, 587973195, 2778479214, 668417883, 2226398520, 1464491431, 2792659882, 3481258691, 2339776369, 2747947338, 3000199533,
3712567952, 376206272, 2149616269, 985682501, 865295391, 1812641626, 567425379, 1468520640, 2273677177, 2267568076, 3898328230, 898149034,
3750298043, 394538907, 4101461357, 2781824777, 2719406676, 3415420393, 122661889, 1452536307, 1463257506, 2874481787, 2250093815, 1439068642,
597070280, 1439076517, 4207797347, 2579732532, 3704826787, 3847236064, 4155289003, 990963026, 2602619627, 701644802, 3629646548, 1110000288,
3609356614, 2748019645, 638526248, 3265491895, 2839687161, 913026615, 2748040592, 975131382, 83378202, 4236013846, 764917668, 1887262417], dtype=uint32),
624,
0,
0.0)
>>> prng.random_sample()
0.20598058788141316
>>> prng.random_sample()
0.6864005375257146
>>> prng.random_sample()
0.08407651896523582
>>> prng.set_state(state)
>>> prng.random_sample()
0.20598058788141316
>>> prng.random_sample()
0.6864005375257146
You can also pickle RandomState objects. We implemented this using the get_state() data, so it will reliably reproduce the state of the PRNG. Depending on exactly what you want to do (you don't say), this is frequently the most convenient thing to do rather than mucking about with get_state() and set_state() manually.
>>> import cPickle
>>> pickled = cPickle.dumps(prng)
>>> prng.random_sample()
0.08407651896523582
>>> prng.random_sample()
0.3501860271954601
>>> prng2 = cPickle.loads(pickled)
>>> prng2.random_sample()
0.08407651896523582
>>> prng2.random_sample()
0.3501860271954601
When people need a random seed that can be recorded, people usually use the system time as a random seed. This means your program will act differently each time it is run, but can be saved and captured. Why don't you try that out?
If you don't want to do that for some reason, use the null version, numpy.random.seed(seed=None), then get a random number from it, then set the seed to that new random number.
You can't… but there's really no good reason to do so. Unless you're actually trying to reproduce the behavior of seed, rather than put the RNG into a repeatable state, you're trying to add an extra level of indirection for no reason.
If you want to stash and restore the RandomState, do that, using the get_state() and set_state() functions.
If you really want to use seed instead, you can just use np.random to generate a random seed (e.g., via random_integers(0, 255, SOME_LENGTH)), which you can stash and reuse later. But there's not much reason to do that.
Or, of course, you can call Python's os.urandom to create a seed the same way NumPy does by default. Note that the docs explicitly say that:
If
seedisNone, thenRandomStatewill try to "read date from/dev/urandom(or the Windows analogue) if available or seed from the clock otherwise.
But again, there's not much reason to do that either. (Also, it isn't documented how much randomness it gets from urandom, so there's always the risk that you'll be seeding it with less random data than it normally uses, or wastefully gathering too much.)
According to the docs, when seed is None, numpy tries to read from /dev/urandom, so why not just read a value from /dev/urandom, save it, and pass it to numpy.random.RandomState?
EDIT:
The internal state can be get and set via get_state and set_state, respectively. So, to recover the initial state, one would do something like this:
>>> import numpy
>>> r = numpy.random.RandomState()
>>> saved_state = r.get_state()
>>> r.rand()
0.9091545657342729
>>> r.rand()
0.9677739782319564
>>> r.rand()
0.5656156400920441
>>> r.set_state(saved_state)
>>> r.rand()
0.9091545657342729
>>> r.rand()
0.9677739782319564
>>> r.rand()
0.5656156400920441
>>>
When seed is None, numpy doesn't pick a "new random seed" and call seed() with it. It reads 624 * sizeof(long) bytes (~ 2.5KB) from /dev/urandom and uses those values to populate the state struct. When you call seed() without arguments, numpy never actually "chooses" a "random seed". Therefore, it's not possible to recover it.
If you want you can also save it in a json file and then unpack it and then use it again. Since numpy stuff can't be serialized you need to serialize it yourself but its not that bad:
One file:
import json
import numpy as np
def put_numpy_seed_in_json_dic(results):
(rnd0,rnd1,rnd2,rnd3,rnd4) = np.random.get_state()
rnd1 = [int(number) for number in rnd1]
rand_seed = (rnd0,rnd1,rnd2,rnd3,rnd4)
results['rand_seed'] = rand_seed
return results
def get_numpy_seed(results):
(rnd0,rnd1,rnd2,rnd3,rnd4) = results['rand_seed']
rnd1 = [np.uint32(number) for number in rnd1]
rand_seed = (rnd0,rnd1,rnd2,rnd3,rnd4)
return rand_seed
then run it to save the seed:
import json
import numpy as np
import my_rand_lib as mr
results = {'rand_seed':None}
results = mr.put_numpy_seed_in_json_dic(results)
print np.random.rand(1)
print np.random.rand(1)
print np.random.rand(1)
fpath = './rand_seed_file'
with open(fpath,'w+') as f:
json.dump(results,f)
print '... doing other stuff'
with open(fpath,'r+') as f:
results2 = json.load(f)
print 'other ',np.random.rand(1)
print 'other ',np.random.rand(1)
print 'other ',np.random.rand(1)
print '... done doing stuff'
rand_seed = mr.get_numpy_seed(results2)
np.random.set_state(rand_seed)
print np.random.rand(1)
print np.random.rand(1)
print np.random.rand(1)
and if you don't want to generate a seed everytime you run it you can have:
import json
import numpy as np
import my_rand_lib as mr
fpath = './rand_seed_file'
with open(fpath,'r+') as f:
results2 = json.load(f)
rand_seed = mr.get_numpy_seed(results2)
np.random.set_state(rand_seed)
print np.random.rand(1)
print np.random.rand(1)
print np.random.rand(1)
I tried this on a remote server and I always get matching random numbers:
[ 0.90741273]
[ 0.6861296]
[ 0.21714398]
not sure if this is interesting but this was the seed (which is a tuple):
{"rand_seed": ["MT19937", [3244492226, 4276548057, 571402114, 3235873143, 4078239958, 1440625038, 4042777784, 3400010150, 1164584760, 271139028, 1264217608, 1403324904, 234696259, 623484078, 3424719234, 3896351743, 1818071683, 3077380191, 2989066157, 3828180331, 2032001745, 1137603205, 1993713826, 873523654, 3267461254, 2964954176, 3217679339, 4079232021, 1182272168, 402998421, 968119626, 2151162455, 2550226639, 3522780791, 245256811, 2866158388, 587411937, 2836234133, 3485394274, 1767143488, 3772379711, 1244725495, 1061026769, 2544419920, 3963050848, 232749713, 2084368489, 1990090546, 2883903063, 174001222, 2569537698, 517341511, 2366955295, 1830324490, 2388090514, 1637855850, 1383101875, 2719629528, 885528387, 7941101, 2769663894, 2704541593, 3129289945, 2681434614, 3308402481, 2161196492, 2896442132, 1474561199, 156414990, 2934014108, 2740454316, 4029663532, 2903418479, 118978587, 3095335574, 1044532364, 2629619463, 623783821, 3172307947, 2539001597, 2020636966, 404303542, 373288588, 289388097, 1050356390, 1126919064, 474676333, 2156863001, 92975776, 1204572119, 1341956590, 4284155262, 3380981209, 1268302262, 835613316, 623125230, 1150083001, 3444902937, 2318349536, 2881496834, 393068269, 28626933, 2931354423, 2014174400, 4212996966, 3105086458, 74404022, 413795342, 3782258177, 3626466932, 1932129332, 3538419256, 943472124, 963175815, 4076955699, 52410025, 318657184, 839799912, 2150435130, 3187525421, 2124551508, 3930704180, 2375548757, 497820208, 422355274, 260159836, 3437157934, 1301403840, 4057357702, 3217300631, 2910194797, 1972036860, 624838554, 3418367281, 3823714808, 1342594222, 3874939587, 3578421466, 3997730187, 751930224, 801189513, 1225089722, 910752086, 1415351761, 4287089458, 224210780, 643596696, 1030838729, 1924676141, 2579935013, 32904138, 2486616018, 1665731347, 642496995, 577928776, 4119274366, 1438990597, 885648199, 2401966414, 1937630298, 2029522084, 3823943785, 1652388617, 242507028, 163957584, 197993457, 3003700508, 2357598705, 479742798, 2159530434, 2641855048, 1153321528, 458640940, 1364908158, 3931878737, 3754891907, 733317650, 3631844997, 209681576, 780025499, 217109730, 2659949782, 164210317, 2234081627, 2798187303, 3793035212, 622613442, 4027945659, 1264924240, 3755962138, 168637328, 4193297896, 593711399, 2018193001, 696136156, 3343926759, 3938753383, 3549915312, 2049590636, 1732826453, 3770804132, 1544263650, 3623494103, 1454784121, 860580298, 1336846278, 3298403325, 4156569419, 51196786, 3398541940, 717201402, 1418590160, 3407195989, 293192063, 3871127471, 963318294, 3177164855, 2248856336, 2363561954, 2122436074, 3083439454, 331898151, 3489466823, 1480231253, 3727404028, 1942269624, 3342915239, 2451833278, 1279324699, 3678779848, 494256563, 170826038, 3200966622, 3284372389, 3798475074, 191206256, 1112201427, 3959301392, 43618741, 1358008929, 2972254642, 2250013335, 659600256, 720199815, 1355589829, 1511937267, 2090180739, 2779086170, 704140912, 1354505400, 4106508219, 4130987887, 1135113560, 3310205054, 2559493616, 3994237157, 2449530906, 1017478859, 2475414025, 260408932, 3882314025, 3169908095, 1431718224, 755730563, 4129813635, 482751982, 42657908, 2418940148, 2380660631, 3596648617, 2668040386, 3700947086, 1235361153, 4212839143, 2803192914, 679783840, 1396721631, 3549531060, 3714188947, 1582886984, 3930587164, 1787845200, 1878170563, 3998685888, 275016726, 1362149445, 1784854500, 3413367687, 999979145, 30464988, 1781846287, 2052179802, 614372595, 1795389478, 3837746383, 1716252322, 1496633789, 1913960414, 3824749341, 745150948, 2990885936, 3557188824, 1853716952, 226442384, 3881419361, 3877508921, 2125849259, 3725330620, 4249819850, 1866002740, 3954375926, 1263697298, 2359110923, 3704149399, 3915156522, 2720534920, 2240262865, 1298116022, 2430494738, 3106481019, 1118448263, 3386525375, 3850025930, 947096317, 2014058358, 2943385566, 1639655978, 824538918, 2893393554, 190010755, 918084027, 4197568458, 2308675470, 3969533604, 823650146, 3971685975, 3959021418, 2335451148, 3651109937, 3536101054, 2028026981, 1042621858, 2093418547, 3332527479, 345797902, 1962843497, 1651609280, 849683942, 701440541, 3001603849, 2547855201, 2847179356, 2686463194, 2556105058, 2957249371, 4122354156, 4095666057, 3269707747, 2075948426, 4189148196, 59188700, 1425136277, 4010662242, 403095998, 2435933607, 3254626634, 320429604, 921618676, 4179054005, 1590495757, 362965764, 3892792894, 4264771139, 300303781, 4194594842, 1773582295, 1792749320, 3114744569, 3059831369, 543108826, 605116437, 1206221920, 3763708911, 3474933214, 933590768, 4096747554, 2732890014, 1180321103, 3174872523, 2361419553, 303084740, 3438967187, 829657141, 3976738932, 3250508727, 2965752967, 2766618501, 2610047728, 3913791738, 2383381107, 2911412379, 2570048205, 1059652767, 1105153800, 258287599, 1366361775, 1043101709, 4136777479, 4002476750, 2242511114, 1937386895, 2318776696, 3919577988, 819932046, 2154232126, 2359937340, 1707529303, 1430709021, 57940224, 2463543918, 439698027, 2154236676, 2989369870, 2711983380, 1243586438, 1648109179, 234677646, 1369164631, 3246772730, 1150951970, 707111532, 2641066313, 1561023105, 2352529521, 3905609297, 3758075920, 4124559541, 3768803924, 3443976002, 2619333832, 3399759018, 2295667887, 4126858561, 3139541980, 2382271429, 4033423715, 648775734, 2777131955, 1929238235, 2146942632, 1115329972, 3985641642, 2007135435, 3551753547, 2967740448, 4196112540, 61581572, 886344810, 4097187928, 1166916633, 3890455280, 1473584306, 1440678763, 2848991175, 2493980496, 3967544385, 1757152663, 52315252, 2476642029, 2727074449, 4197000746, 2878883929, 144032869, 3517610268, 3758074755, 164078969, 4288210033, 1130401207, 2376285572, 3726677017, 2021546352, 2763363362, 791950895, 1834778577, 3067448324, 2618082688, 4194263605, 84230440, 624358904, 3203686228, 2014115933, 1844566018, 314698511, 1096366940, 1413533306, 2490690918, 3524310116, 3912232452, 3595400103, 2104097721, 2277699865, 2127808758, 890104002, 4261780514, 3943759279, 2421596910, 3462302371, 3114202694, 3301664792, 3958641805, 3828288008, 3138631754, 2707054121, 873889048, 360096040, 1277036249, 310404450, 1841086653, 1324064291, 1069123460, 3667889879, 3549162319, 4105010862, 3802778145, 1818048305, 3083126999, 3810922140, 2364932315, 2667079274, 3034477663, 1142598277, 2129656233, 2900596493, 1771721766, 2091125900, 2024931777, 2186939139, 4292757779, 3168005700, 2700706967, 2033965363, 2815886839, 1936909550, 1018210446, 2494829103, 3182190430, 4070030839, 3878343946, 3290625485, 2885062721, 3427598831, 3748858811, 2021454997, 2926497731, 3462334646, 747641905, 2870980834, 1072943394, 370913272, 519334913, 3099507262, 698616436, 3884871568, 2530196197, 223690634, 1816574877, 2872502342, 3629966511, 4040316403, 400367036, 1898479168, 1795033191, 4090946019, 938326326, 1509105095, 886381170, 4207241822, 2919702734, 1437184594, 2765872952, 561764883, 3441440757, 1219765705, 209412518, 1098738818, 3782425126, 3113624586, 3302772981, 1213966890, 4292826280, 4109015079, 1949958581, 320991923, 1070765942, 2002780881, 3364869673, 3039286974, 1824574474, 1266616388, 703321141, 2004303453, 1284326590, 728587648, 427042526, 2160662521, 2783764788, 3053336315, 3542331332, 2881174731, 4160514263, 3326878203, 4139791808, 2639767143, 3144886711, 480269073, 2318151636, 3594165209, 1629301762, 1786754501, 1157007028, 1415023980, 172137771, 3444342355, 3889095376], 624, 0, 0.0]}
