Pyro: A failure of reproduction in AIR example

[Minju-Jung]

I tried to reproduce the result of AIR reported in http://pyro.ai/examples/air.html.
I used the default setting of main.py as follows:
python main.py --progress-every 100 --eval-every 100 --viz --cuda

However, the count accuracy did not increased more than 33%.
In case of zero digit, the network successfully estimated the number of digits as zero, but the network wrongly estimated the number of digits as three in the other cases (one and two digits).

Could somebody help me to solve the problem?

i=10000, epochs=10.67, elapsed=0.11, elbo=-622.77
i=10000, accuracy=0.19103333333333333, counts=[[11430, 5505, 2139, 938], [152, 32, 15, 19858], [2, 0, 0, 19929]]
i=20000, epochs=21.33, elapsed=0.22, elbo=-630.32
i=20000, accuracy=0.26921666666666666, counts=[[16150, 3750, 112, 0], [64, 3, 1, 19989], [0, 0, 0, 19931]]
i=30000, epochs=32.00, elapsed=0.33, elbo=-643.42
i=30000, accuracy=0.32221666666666665, counts=[[19331, 681, 0, 0], [86, 2, 0, 19969], [0, 0, 0, 19931]]
i=40000, epochs=42.67, elapsed=0.44, elbo=-641.45
i=40000, accuracy=0.32425, counts=[[19455, 557, 0, 0], [44, 0, 0, 20013], [0, 0, 0, 19931]]

[martinjankowiak]

have you tried the settings explicitly listed at the end of the tutorial?

On Dec 19, 2017 8:48 AM, "Minju Jung" [e-mail] wrote:

I tried to reproduce the result of AIR reported in
http://pyro.ai/examples/air.html.
I used the default setting of main.py as follows:
python main.py --progress-every 100 --eval-every 100 --viz --cuda

However, the count accuracy did not increased more than 33%.
In case of zero digit, the network successfully estimated the number of
digits as zero, but the network wrongly estimated the number of digits as
three in the other cases (one and two digits).

Could somebody help me to solve the problem?

i=100, epochs=0.11, elapsed=0.00, elbo=-433.93
i=100, accuracy=0.28875, counts=[[9785, 5135, 2653, 2439], [10370, 5232,
2459, 1996], [10740, 5275, 2308, 1608]]
i=200, epochs=0.21, elapsed=0.00, elbo=-460.19
i=200, accuracy=0.29096666666666665, counts=[[9935, 5196, 2579, 2302],
[10521, 5422, 2470, 1644], [11134, 5472, 2101, 1224]]
i=300, epochs=0.32, elapsed=0.00, elbo=-485.51
i=300, accuracy=0.29046666666666665, counts=[[10009, 5279, 2553, 2171],
[10885, 5523, 2285, 1364], [11840, 5382, 1896, 813]]
i=400, epochs=0.43, elapsed=0.00, elbo=-504.17
i=400, accuracy=0.2906166666666667, counts=[[9996, 5423, 2636, 1957],
[11073, 5555, 2239, 1190], [11736, 5598, 1886, 711]]
i=500, epochs=0.53, elapsed=0.01, elbo=-450.12
i=500, accuracy=0.3021, counts=[[10350, 5300, 2575, 1787], [10766, 5607,
2341, 1343], [10954, 5850, 2169, 958]]
i=600, epochs=0.64, elapsed=0.01, elbo=-483.70
i=600, accuracy=0.3065833333333333, counts=[[10188, 5302, 2578, 1944],
[9903, 5495, 2601, 2058], [9681, 5413, 2712, 2125]]
i=700, epochs=0.75, elapsed=0.01, elbo=-499.12
i=700, accuracy=0.30533333333333335, counts=[[10040, 5348, 2646, 1978],
[8743, 5223, 2908, 3183], [7645, 4732, 3057, 4497]]
i=800, epochs=0.85, elapsed=0.01, elbo=-509.91
i=800, accuracy=0.27058333333333334, counts=[[9737, 5412, 2748, 2115],
[7204, 4340, 2779, 5734], [5256, 3200, 2158, 9317]]
i=900, epochs=0.96, elapsed=0.01, elbo=-524.70
i=900, accuracy=0.2242, counts=[[9605, 5376, 2775, 2256], [5558, 3048,
1829, 9622], [2971, 1308, 799, 14853]]
i=1000, epochs=1.07, elapsed=0.01, elbo=-538.22
i=1000, accuracy=0.20658333333333334, counts=[[9470, 5363, 2804, 2375],
[4778, 2428, 1522, 11329], [2045, 848, 497, 16541]]
i=1100, epochs=1.17, elapsed=0.01, elbo=-531.56
i=1100, accuracy=0.19363333333333332, counts=[[9445, 5315, 2794, 2458],
[3988, 1868, 1209, 12992], [1522, 535, 305, 17569]]
i=1200, epochs=1.28, elapsed=0.01, elbo=-529.06
i=1200, accuracy=0.18973333333333334, counts=[[9490, 5397, 2861, 2264],
[3642, 1691, 1032, 13692], [1217, 383, 203, 18128]]
i=1300, epochs=1.39, elapsed=0.01, elbo=-555.53
i=1300, accuracy=0.18466666666666667, counts=[[9478, 5368, 2909, 2257],
[3183, 1435, 877, 14562], [886, 265, 167, 18613]]
i=1400, epochs=1.49, elapsed=0.02, elbo=-537.41
i=1400, accuracy=0.18071666666666666, counts=[[9259, 5493, 2897, 2363],
[2996, 1445, 903, 14713], [756, 288, 139, 18748]]
i=1500, epochs=1.60, elapsed=0.02, elbo=-554.44
i=1500, accuracy=0.18023333333333333, counts=[[9354, 5487, 2895, 2276],
[2762, 1334, 783, 15178], [644, 201, 126, 18960]]
i=1600, epochs=1.71, elapsed=0.02, elbo=-544.14
i=1600, accuracy=0.16543333333333332, counts=[[9017, 5427, 3013, 2555],
[2172, 861, 428, 16596], [432, 108, 48, 19343]]
i=1700, epochs=1.81, elapsed=0.02, elbo=-588.13
i=1700, accuracy=0.16085, counts=[[8920, 5497, 2976, 2619], [1920, 701,
417, 17019], [353, 74, 30, 19474]]
i=1800, epochs=1.92, elapsed=0.02, elbo=-555.77
i=1800, accuracy=0.16401666666666667, counts=[[9045, 5407, 2985, 2575],
[1812, 756, 470, 17019], [332, 82, 40, 19477]]
i=1900, epochs=2.03, elapsed=0.02, elbo=-563.41
i=1900, accuracy=0.1654, counts=[[9171, 5404, 3037, 2400], [1806, 709,
488, 17054], [275, 70, 44, 19542]]
i=2000, epochs=2.13, elapsed=0.02, elbo=-572.18
i=2000, accuracy=0.1599, counts=[[8996, 5726, 3036, 2254], [1627, 565,
343, 17522], [257, 60, 33, 19581]]
i=2100, epochs=2.24, elapsed=0.02, elbo=-576.92
i=2100, accuracy=0.16115, counts=[[9211, 5415, 3044, 2342], [1479, 440,
218, 17920], [214, 38, 18, 19661]]
i=2200, epochs=2.35, elapsed=0.02, elbo=-556.89
i=2200, accuracy=0.1561, counts=[[8973, 5374, 3068, 2597], [1241, 377,
236, 18203], [147, 28, 16, 19740]]
i=2300, epochs=2.45, elapsed=0.03, elbo=-585.29
i=2300, accuracy=0.15621666666666667, counts=[[8997, 5388, 2970, 2657],
[1164, 367, 169, 18357], [163, 22, 9, 19737]]
i=2400, epochs=2.56, elapsed=0.03, elbo=-575.51
i=2400, accuracy=0.15325, counts=[[8873, 5224, 3032, 2883], [1168, 313,
176, 18400], [112, 12, 9, 19798]]
i=2500, epochs=2.67, elapsed=0.03, elbo=-545.33
i=2500, accuracy=0.15605, counts=[[8966, 5437, 2997, 2612], [1253, 382,
182, 18240], [151, 17, 15, 19748]]
i=2600, epochs=2.77, elapsed=0.03, elbo=-586.10
i=2600, accuracy=0.15653333333333333, counts=[[8985, 5419, 2995, 2613],
[1180, 396, 233, 18248], [111, 26, 11, 19783]]
i=2700, epochs=2.88, elapsed=0.03, elbo=-576.12
i=2700, accuracy=0.15848333333333334, counts=[[9097, 5459, 2914, 2542],
[1171, 393, 199, 18294], [109, 25, 19, 19778]]
i=2800, epochs=2.99, elapsed=0.03, elbo=-573.36
i=2800, accuracy=0.16275, counts=[[9345, 5556, 2931, 2180], [1177, 404,
244, 18232], [102, 28, 16, 19785]]
i=2900, epochs=3.09, elapsed=0.03, elbo=-585.09
i=2900, accuracy=0.16013333333333332, counts=[[9333, 5448, 2978, 2253],
[985, 263, 178, 18631], [84, 16, 12, 19819]]
i=3000, epochs=3.20, elapsed=0.03, elbo=-593.55
i=3000, accuracy=0.15528333333333333, counts=[[9056, 5554, 3018, 2384],
[951, 255, 141, 18710], [73, 14, 6, 19838]]
i=3100, epochs=3.31, elapsed=0.03, elbo=-562.91
i=3100, accuracy=0.15146666666666667, counts=[[8833, 5566, 2977, 2636],
[789, 244, 125, 18899], [59, 7, 11, 19854]]
i=3200, epochs=3.41, elapsed=0.04, elbo=-566.64
i=3200, accuracy=0.1532, counts=[[9001, 5568, 3162, 2281], [736, 184, 100,
19037], [48, 5, 7, 19871]]
i=3300, epochs=3.52, elapsed=0.04, elbo=-594.62
i=3300, accuracy=0.15438333333333334, counts=[[9062, 5732, 3103, 2115],
[754, 198, 117, 18988], [48, 13, 3, 19867]]
i=3400, epochs=3.63, elapsed=0.04, elbo=-597.69
i=3400, accuracy=0.15765, counts=[[9236, 5781, 3083, 1912], [789, 212,
126, 18930], [56, 6, 11, 19858]]
i=3500, epochs=3.73, elapsed=0.04, elbo=-578.24
i=3500, accuracy=0.1599, counts=[[9347, 5936, 2999, 1730], [742, 239, 114,
18962], [57, 2, 8, 19864]]
i=3600, epochs=3.84, elapsed=0.04, elbo=-602.63
i=3600, accuracy=0.16173333333333334, counts=[[9468, 6059, 2941, 1544],
[733, 228, 123, 18973], [60, 9, 8, 19854]]
i=3700, epochs=3.95, elapsed=0.04, elbo=-600.70
i=3700, accuracy=0.16193333333333335, counts=[[9492, 5765, 2888, 1867],
[710, 218, 113, 19016], [52, 12, 6, 19861]]
i=3800, epochs=4.05, elapsed=0.04, elbo=-616.22
i=3800, accuracy=0.15945, counts=[[9372, 5862, 2991, 1787], [670, 187,
111, 19089], [52, 7, 8, 19864]]
i=3900, epochs=4.16, elapsed=0.04, elbo=-582.18
i=3900, accuracy=0.16, counts=[[9393, 5856, 2963, 1800], [689, 205, 122,
19041], [42, 5, 2, 19882]]
i=4000, epochs=4.27, elapsed=0.04, elbo=-569.28
i=4000, accuracy=0.15486666666666668, counts=[[9172, 5661, 2998, 2181],
[562, 116, 82, 19297], [29, 1, 4, 19897]]
i=4100, epochs=4.37, elapsed=0.04, elbo=-579.48
i=4100, accuracy=0.15646666666666667, counts=[[9213, 5909, 3024, 1866],
[583, 171, 106, 19197], [27, 8, 4, 19892]]
i=4200, epochs=4.48, elapsed=0.05, elbo=-612.26
i=4200, accuracy=0.16058333333333333, counts=[[9464, 6132, 2957, 1459],
[511, 164, 128, 19254], [21, 6, 7, 19897]]
i=4300, epochs=4.59, elapsed=0.05, elbo=-599.60
i=4300, accuracy=0.16056666666666666, counts=[[9444, 6116, 2929, 1523],
[479, 183, 112, 19283], [28, 6, 7, 19890]]
i=4400, epochs=4.69, elapsed=0.05, elbo=-600.47
i=4400, accuracy=0.16593333333333332, counts=[[9781, 6135, 2768, 1328],
[562, 172, 121, 19202], [24, 4, 3, 19900]]
i=4500, epochs=4.80, elapsed=0.05, elbo=-614.06
i=4500, accuracy=0.16526666666666667, counts=[[9723, 6110, 2785, 1394],
[541, 188, 186, 19142], [22, 9, 5, 19895]]
i=4600, epochs=4.91, elapsed=0.05, elbo=-584.82
i=4600, accuracy=0.16175, counts=[[9516, 6128, 2894, 1474], [498, 184,
153, 19222], [26, 2, 5, 19898]]
i=4700, epochs=5.01, elapsed=0.05, elbo=-599.46
i=4700, accuracy=0.15968333333333334, counts=[[9421, 6290, 2919, 1382],
[436, 156, 124, 19341], [21, 6, 4, 19900]]
i=4800, epochs=5.12, elapsed=0.05, elbo=-581.86
i=4800, accuracy=0.16358333333333333, counts=[[9628, 6003, 2933, 1448],
[475, 180, 135, 19267], [25, 3, 7, 19896]]
i=4900, epochs=5.23, elapsed=0.05, elbo=-592.77
i=4900, accuracy=0.1618, counts=[[9536, 6223, 2952, 1301], [472, 169, 131,
19285], [24, 4, 3, 19900]]
i=5000, epochs=5.33, elapsed=0.05, elbo=-610.56
i=5000, accuracy=0.15631666666666666, counts=[[9245, 5922, 3009, 1836],
[388, 133, 98, 19438], [21, 8, 1, 19901]]
i=5100, epochs=5.44, elapsed=0.06, elbo=-622.01
i=5100, accuracy=0.15105, counts=[[8960, 5611, 3037, 2404], [423, 98, 61,
19475], [14, 1, 5, 19911]]
i=5200, epochs=5.55, elapsed=0.06, elbo=-618.06
i=5200, accuracy=0.1544, counts=[[9143, 5804, 3039, 2026], [419, 121, 87,
19430], [20, 3, 0, 19908]]
i=5300, epochs=5.65, elapsed=0.06, elbo=-605.74
i=5300, accuracy=0.15703333333333333, counts=[[9318, 5606, 2971, 2117],
[410, 101, 69, 19477], [8, 5, 3, 19915]]
i=5400, epochs=5.76, elapsed=0.06, elbo=-622.64
i=5400, accuracy=0.16155, counts=[[9575, 6006, 2864, 1567], [403, 117, 62,
19475], [22, 5, 1, 19903]]
i=5500, epochs=5.87, elapsed=0.06, elbo=-590.45
i=5500, accuracy=0.16025, counts=[[9495, 6131, 2999, 1387], [400, 117, 66,
19474], [12, 2, 3, 19914]]
i=5600, epochs=5.97, elapsed=0.06, elbo=-609.16
i=5600, accuracy=0.162, counts=[[9607, 6168, 2993, 1244], [406, 109, 84,
19458], [8, 3, 4, 19916]]
i=5700, epochs=6.08, elapsed=0.06, elbo=-626.69
i=5700, accuracy=0.15965, counts=[[9484, 5992, 2970, 1566], [363, 93, 64,
19537], [10, 5, 2, 19914]]
i=5800, epochs=6.19, elapsed=0.06, elbo=-590.29
i=5800, accuracy=0.15441666666666667, counts=[[9180, 5737, 3027, 2068],
[338, 84, 65, 19570], [12, 2, 1, 19916]]
i=5900, epochs=6.29, elapsed=0.06, elbo=-612.98
i=5900, accuracy=0.16263333333333332, counts=[[9665, 6120, 2895, 1332],
[351, 93, 47, 19566], [12, 5, 0, 19914]]
i=6000, epochs=6.40, elapsed=0.07, elbo=-591.39
i=6000, accuracy=0.16113333333333332, counts=[[9585, 6048, 2875, 1504],
[351, 81, 56, 19569], [11, 2, 2, 19916]]
i=6100, epochs=6.51, elapsed=0.07, elbo=-610.47
i=6100, accuracy=0.156, counts=[[9288, 6095, 3017, 1612], [292, 69, 57,
19639], [14, 0, 3, 19914]]
i=6200, epochs=6.61, elapsed=0.07, elbo=-604.13
i=6200, accuracy=0.15841666666666668, counts=[[9442, 5705, 2963, 1902],
[298, 62, 36, 19661], [11, 1, 1, 19918]]
i=6300, epochs=6.72, elapsed=0.07, elbo=-601.21
i=6300, accuracy=0.15306666666666666, counts=[[9113, 5754, 2942, 2203],
[288, 67, 33, 19669], [8, 1, 4, 19918]]
i=6400, epochs=6.83, elapsed=0.07, elbo=-592.04
i=6400, accuracy=0.1495, counts=[[8913, 5498, 3037, 2564], [238, 55, 28,
19736], [10, 1, 2, 19918]]
i=6500, epochs=6.93, elapsed=0.07, elbo=-615.72
i=6500, accuracy=0.1465, counts=[[8752, 5769, 3001, 2490], [256, 37, 28,
19736], [8, 0, 1, 19922]]
i=6600, epochs=7.04, elapsed=0.07, elbo=-620.83
i=6600, accuracy=0.1447, counts=[[8632, 5342, 2984, 3054], [233, 49, 22,
19753], [4, 2, 1, 19924]]
i=6700, epochs=7.15, elapsed=0.07, elbo=-603.72
i=6700, accuracy=0.14251666666666668, counts=[[8505, 5297, 2915, 3295],
[194, 45, 26, 19792], [3, 1, 1, 19926]]
i=6800, epochs=7.25, elapsed=0.07, elbo=-621.48
i=6800, accuracy=0.14505, counts=[[8655, 5350, 2846, 3161], [195, 48, 29,
19785], [6, 0, 0, 19925]]
i=6900, epochs=7.36, elapsed=0.08, elbo=-607.07
i=6900, accuracy=0.14005, counts=[[8375, 5167, 2926, 3544], [186, 26, 18,
19827], [7, 1, 2, 19921]]
i=7000, epochs=7.47, elapsed=0.08, elbo=-617.85
i=7000, accuracy=0.14365, counts=[[8587, 5168, 2913, 3344], [179, 32, 23,
19823], [2, 1, 0, 19928]]
i=7100, epochs=7.57, elapsed=0.08, elbo=-611.23
i=7100, accuracy=0.14203333333333334, counts=[[8489, 5289, 3061, 3173],
[171, 32, 17, 19837], [3, 1, 1, 19926]]
i=7200, epochs=7.68, elapsed=0.08, elbo=-597.13
i=7200, accuracy=0.14706666666666668, counts=[[8788, 6007, 3059, 2158],
[191, 35, 18, 19813], [0, 1, 1, 19929]]
i=7300, epochs=7.79, elapsed=0.08, elbo=-614.46
i=7300, accuracy=0.15485, counts=[[9257, 6164, 3082, 1509], [188, 34, 22,
19813], [4, 0, 0, 19927]]
i=7400, epochs=7.89, elapsed=0.08, elbo=-630.82
i=7400, accuracy=0.15018333333333334, counts=[[8977, 6046, 3120, 1869],
[177, 34, 20, 19826], [3, 0, 0, 19928]]
i=7500, epochs=8.00, elapsed=0.08, elbo=-619.98
i=7500, accuracy=0.14396666666666666, counts=[[8604, 5934, 3166, 2308],
[168, 34, 28, 19827], [5, 0, 0, 19926]]
i=7600, epochs=8.11, elapsed=0.08, elbo=-628.65
i=7600, accuracy=0.15188333333333334, counts=[[9087, 5848, 3053, 2024],
[191, 26, 13, 19827], [5, 0, 0, 19926]]
i=7700, epochs=8.21, elapsed=0.08, elbo=-596.34
i=7700, accuracy=0.1518, counts=[[9071, 5881, 3013, 2047], [201, 37, 21,
19798], [7, 0, 0, 19924]]
i=7800, epochs=8.32, elapsed=0.09, elbo=-635.09
i=7800, accuracy=0.1635, counts=[[9760, 6060, 2759, 1433], [216, 50, 30,
19761], [3, 0, 0, 19928]]
i=7900, epochs=8.43, elapsed=0.09, elbo=-625.90
i=7900, accuracy=0.16215, counts=[[9690, 5793, 2784, 1745], [259, 37, 16,
19745], [3, 1, 2, 19925]]
i=8000, epochs=8.53, elapsed=0.09, elbo=-621.57
i=8000, accuracy=0.1604, counts=[[9586, 5361, 2694, 2371], [209, 37, 35,
19776], [2, 0, 1, 19928]]
i=8100, epochs=8.64, elapsed=0.09, elbo=-618.93
i=8100, accuracy=0.1645, counts=[[9834, 5456, 2665, 2057], [209, 35, 22,
19791], [9, 0, 1, 19921]]
i=8200, epochs=8.75, elapsed=0.09, elbo=-605.28
i=8200, accuracy=0.15953333333333333, counts=[[9540, 5530, 2795, 2147],
[209, 32, 19, 19797], [7, 0, 0, 19924]]
i=8300, epochs=8.85, elapsed=0.09, elbo=-627.01
i=8300, accuracy=0.16393333333333332, counts=[[9794, 5645, 2665, 1908],
[213, 42, 23, 19779], [6, 0, 0, 19925]]
i=8400, epochs=8.96, elapsed=0.09, elbo=-623.80
i=8400, accuracy=0.17005, counts=[[10156, 5689, 2532, 1635], [223, 46, 24,
19764], [3, 1, 1, 19926]]
i=8500, epochs=9.07, elapsed=0.09, elbo=-631.31
i=8500, accuracy=0.1709, counts=[[10213, 5499, 2619, 1681], [232, 40, 31,
19754], [10, 0, 1, 19920]]
i=8600, epochs=9.17, elapsed=0.09, elbo=-615.25
i=8600, accuracy=0.17445, counts=[[10426, 5679, 2364, 1543], [218, 40, 36,
19763], [8, 0, 1, 19922]]
i=8700, epochs=9.28, elapsed=0.10, elbo=-617.79
i=8700, accuracy=0.17576666666666665, counts=[[10504, 5660, 2462, 1386],
[221, 41, 33, 19762], [6, 0, 1, 19924]]
i=8800, epochs=9.39, elapsed=0.10, elbo=-611.90
i=8800, accuracy=0.17246666666666666, counts=[[10317, 5356, 2513, 1826],
[208, 30, 28, 19791], [7, 1, 1, 19922]]
i=8900, epochs=9.49, elapsed=0.10, elbo=-630.25
i=8900, accuracy=0.16506666666666667, counts=[[9880, 4972, 2487, 2673],
[207, 23, 20, 19807], [2, 1, 1, 19927]]
i=9000, epochs=9.60, elapsed=0.10, elbo=-622.22
i=9000, accuracy=0.17155, counts=[[10268, 5287, 2430, 2027], [240, 24, 27,
19766], [13, 1, 1, 19916]]
i=9100, epochs=9.71, elapsed=0.10, elbo=-632.19
i=9100, accuracy=0.17808333333333334, counts=[[10655, 5474, 2369, 1514],
[215, 30, 28, 19784], [6, 0, 0, 19925]]
i=9200, epochs=9.81, elapsed=0.10, elbo=-639.12
i=9200, accuracy=0.1775, counts=[[10624, 5266, 2359, 1763], [188, 26, 21,
19822], [4, 0, 0, 19927]]
i=9300, epochs=9.92, elapsed=0.10, elbo=-613.00
i=9300, accuracy=0.16958333333333334, counts=[[10149, 5099, 2401, 2363],
[196, 24, 29, 19808], [3, 1, 2, 19925]]
i=9400, epochs=10.03, elapsed=0.10, elbo=-609.39
i=9400, accuracy=0.16775, counts=[[10038, 4912, 2443, 2619], [184, 27, 23,
19823], [6, 0, 0, 19925]]
i=9500, epochs=10.13, elapsed=0.10, elbo=-604.41
i=9500, accuracy=0.16943333333333332, counts=[[10130, 5117, 2467, 2298],
[205, 35, 20, 19797], [8, 0, 1, 19922]]
i=9600, epochs=10.24, elapsed=0.11, elbo=-617.36
i=9600, accuracy=0.1851, counts=[[11072, 5735, 2317, 888], [250, 34, 27,
19746], [10, 0, 0, 19921]]
i=9700, epochs=10.35, elapsed=0.11, elbo=-616.74
i=9700, accuracy=0.18818333333333334, counts=[[11264, 5853, 2157, 738],
[223, 26, 20, 19788], [5, 0, 1, 19925]]
i=9800, epochs=10.45, elapsed=0.11, elbo=-621.91
i=9800, accuracy=0.19258333333333333, counts=[[11537, 5861, 2051, 563],
[217, 16, 12, 19812], [3, 1, 2, 19925]]
i=9900, epochs=10.56, elapsed=0.11, elbo=-624.24
i=9900, accuracy=0.18561666666666668, counts=[[11117, 5812, 2301, 782],
[175, 20, 23, 19839], [7, 0, 0, 19924]]
i=10000, epochs=10.67, elapsed=0.11, elbo=-622.77
i=10000, accuracy=0.19103333333333333, counts=[[11430, 5505, 2139, 938],
[152, 32, 15, 19858], [2, 0, 0, 19929]]
i=10100, epochs=10.77, elapsed=0.11, elbo=-617.29
i=10100, accuracy=0.18836666666666665, counts=[[11284, 5744, 2149, 835],
[175, 18, 22, 19842], [1, 0, 0, 19930]]
i=10200, epochs=10.88, elapsed=0.11, elbo=-625.72
i=10200, accuracy=0.19708333333333333, counts=[[11791, 5800, 1906, 515],
[186, 34, 27, 19810], [4, 0, 0, 19927]]
i=10300, epochs=10.99, elapsed=0.11, elbo=-629.23
i=10300, accuracy=0.19046666666666667, counts=[[11413, 5595, 2098, 906],
[190, 15, 13, 19839], [2, 1, 0, 19928]]
i=10400, epochs=11.09, elapsed=0.11, elbo=-617.79
i=10400, accuracy=0.1888, counts=[[11311, 5653, 2138, 910], [197, 17, 18,
19825], [7, 0, 0, 19924]]
i=10500, epochs=11.20, elapsed=0.12, elbo=-629.83
i=10500, accuracy=0.2008, counts=[[12028, 5576, 1815, 593], [184, 19, 21,
19833], [3, 0, 1, 19927]]
i=10600, epochs=11.31, elapsed=0.12, elbo=-627.51
i=10600, accuracy=0.18845, counts=[[11289, 5470, 2151, 1102], [132, 17,
10, 19898], [3, 0, 1, 19927]]
i=10700, epochs=11.41, elapsed=0.12, elbo=-632.37
i=10700, accuracy=0.19633333333333333, counts=[[11759, 5204, 1958, 1091],
[170, 21, 15, 19851], [3, 0, 0, 19928]]
i=10800, epochs=11.52, elapsed=0.12, elbo=-619.19
i=10800, accuracy=0.20566666666666666, counts=[[12324, 5540, 1721, 427],
[190, 16, 7, 19844], [2, 0, 0, 19929]]
i=10900, epochs=11.63, elapsed=0.12, elbo=-639.19
i=10900, accuracy=0.21111666666666667, counts=[[12640, 5576, 1511, 285],
[206, 27, 5, 19819], [2, 1, 0, 19928]]
i=11000, epochs=11.73, elapsed=0.12, elbo=-610.61
i=11000, accuracy=0.20185, counts=[[12093, 5550, 1825, 544], [167, 18, 10,
19862], [1, 1, 0, 19929]]
i=11100, epochs=11.84, elapsed=0.12, elbo=-617.53
i=11100, accuracy=0.2113, counts=[[12669, 5569, 1464, 310], [172, 9, 9,
19867], [1, 0, 0, 19930]]
i=11200, epochs=11.95, elapsed=0.12, elbo=-619.60
i=11200, accuracy=0.21273333333333333, counts=[[12749, 5613, 1426, 224],
[189, 14, 6, 19848], [1, 0, 1, 19929]]
i=11300, epochs=12.05, elapsed=0.12, elbo=-592.96
i=11300, accuracy=0.20783333333333334, counts=[[12458, 5518, 1634, 402],
[169, 12, 7, 19869], [5, 0, 0, 19926]]
i=11400, epochs=12.16, elapsed=0.13, elbo=-629.99
i=11400, accuracy=0.21375, counts=[[12805, 5545, 1447, 215], [168, 20, 4,
19865], [2, 1, 0, 19928]]
i=11500, epochs=12.27, elapsed=0.13, elbo=-633.36
i=11500, accuracy=0.208, counts=[[12463, 5651, 1592, 306], [174, 17, 9,
19857], [3, 0, 0, 19928]]
i=11600, epochs=12.37, elapsed=0.13, elbo=-641.96
i=11600, accuracy=0.2169, counts=[[12992, 5471, 1332, 217], [218, 22, 9,
19808], [5, 1, 0, 19925]]
i=11700, epochs=12.48, elapsed=0.13, elbo=-637.37
i=11700, accuracy=0.22705, counts=[[13612, 5301, 1012, 87], [179, 10, 6,
19862], [0, 0, 1, 19930]]
i=11800, epochs=12.59, elapsed=0.13, elbo=-618.15
i=11800, accuracy=0.22946666666666668, counts=[[13757, 5256, 933, 66],
[202, 11, 3, 19841], [1, 0, 0, 19930]]
i=11900, epochs=12.69, elapsed=0.13, elbo=-629.00
i=11900, accuracy=0.23408333333333334, counts=[[14031, 5191, 751, 39],
[186, 14, 6, 19851], [1, 0, 0, 19930]]
i=12000, epochs=12.80, elapsed=0.13, elbo=-619.21
i=12000, accuracy=0.23918333333333333, counts=[[14338, 4957, 675, 42],
[183, 13, 8, 19853], [1, 0, 0, 19930]]
i=12100, epochs=12.91, elapsed=0.13, elbo=-626.01
i=12100, accuracy=0.24311666666666668, counts=[[14579, 4835, 578, 20],
[190, 7, 3, 19857], [2, 0, 1, 19928]]
i=12200, epochs=13.01, elapsed=0.13, elbo=-626.78
i=12200, accuracy=0.24751666666666666, counts=[[14840, 4658, 494, 20],
[196, 11, 4, 19846], [5, 0, 0, 19926]]
i=12300, epochs=13.12, elapsed=0.13, elbo=-610.89
i=12300, accuracy=0.24453333333333332, counts=[[14665, 4796, 534, 17],
[183, 7, 5, 19862], [0, 0, 0, 19931]]
i=12400, epochs=13.23, elapsed=0.14, elbo=-633.04
i=12400, accuracy=0.23765, counts=[[14251, 4941, 773, 47], [137, 7, 2,
19911], [1, 0, 1, 19929]]
i=12500, epochs=13.33, elapsed=0.14, elbo=-618.48
i=12500, accuracy=0.24145, counts=[[14482, 4826, 670, 34], [167, 5, 4,
19881], [1, 0, 0, 19930]]
i=12600, epochs=13.44, elapsed=0.14, elbo=-622.23
i=12600, accuracy=0.23443333333333333, counts=[[14058, 5044, 855, 55],
[155, 8, 6, 19888], [2, 0, 0, 19929]]
i=12700, epochs=13.55, elapsed=0.14, elbo=-641.91
i=12700, accuracy=0.24158333333333334, counts=[[14490, 4825, 668, 29],
[159, 5, 1, 19892], [2, 0, 0, 19929]]
i=12800, epochs=13.65, elapsed=0.14, elbo=-625.56
i=12800, accuracy=0.2507666666666667, counts=[[15040, 4531, 429, 12],
[144, 6, 5, 19902], [0, 0, 0, 19931]]
i=12900, epochs=13.76, elapsed=0.14, elbo=-632.85
i=12900, accuracy=0.2618, counts=[[15698, 4063, 247, 4], [150, 10, 3,
19894], [2, 0, 0, 19929]]
i=13000, epochs=13.87, elapsed=0.14, elbo=-631.86
i=13000, accuracy=0.266, counts=[[15952, 3850, 207, 3], [165, 8, 4,
19880], [1, 0, 0, 19930]]
i=13100, epochs=13.97, elapsed=0.14, elbo=-640.80
i=13100, accuracy=0.26065, counts=[[15635, 4105, 268, 4], [148, 4, 2,
19903], [2, 0, 0, 19929]]
i=13200, epochs=14.08, elapsed=0.15, elbo=-636.51
i=13200, accuracy=0.25866666666666666, counts=[[15510, 4219, 278, 5],
[164, 10, 1, 19882], [1, 0, 0, 19930]]
i=13300, epochs=14.19, elapsed=0.15, elbo=-644.96
i=13300, accuracy=0.2569166666666667, counts=[[15412, 4323, 273, 4], [147,
3, 1, 19906], [3, 0, 0, 19928]]
i=13400, epochs=14.29, elapsed=0.15, elbo=-638.02
i=13400, accuracy=0.24823333333333333, counts=[[14888, 4711, 404, 9],
[136, 6, 2, 19913], [2, 0, 0, 19929]]
i=13500, epochs=14.40, elapsed=0.15, elbo=-630.33
i=13500, accuracy=0.24206666666666668, counts=[[14519, 4931, 537, 25],
[114, 5, 2, 19936], [1, 0, 0, 19930]]
i=13600, epochs=14.51, elapsed=0.15, elbo=-629.57
i=13600, accuracy=0.25311666666666666, counts=[[15179, 4426, 395, 12],
[134, 7, 3, 19913], [1, 0, 1, 19929]]
i=13700, epochs=14.61, elapsed=0.15, elbo=-637.02
i=13700, accuracy=0.25983333333333336, counts=[[15585, 4121, 302, 4],
[136, 5, 4, 19912], [0, 0, 0, 19931]]
i=13800, epochs=14.72, elapsed=0.15, elbo=-624.79
i=13800, accuracy=0.2455, counts=[[14723, 4784, 487, 18], [114, 7, 5,
19931], [0, 0, 0, 19931]]
i=13900, epochs=14.83, elapsed=0.15, elbo=-628.12
i=13900, accuracy=0.24116666666666667, counts=[[14461, 4925, 601, 25],
[98, 8, 2, 19949], [0, 0, 1, 19930]]
i=14000, epochs=14.93, elapsed=0.15, elbo=-640.48
i=14000, accuracy=0.23408333333333334, counts=[[14040, 5148, 775, 49],
[103, 5, 2, 19947], [0, 0, 0, 19931]]
i=14100, epochs=15.04, elapsed=0.16, elbo=-625.90
i=14100, accuracy=0.22576666666666667, counts=[[13542, 5336, 1012, 122],
[91, 4, 2, 19960], [2, 0, 0, 19929]]
i=14200, epochs=15.15, elapsed=0.16, elbo=-628.53
i=14200, accuracy=0.22741666666666666, counts=[[13640, 5371, 926, 75],
[84, 5, 3, 19965], [1, 0, 0, 19930]]
i=14300, epochs=15.25, elapsed=0.16, elbo=-629.29
i=14300, accuracy=0.23033333333333333, counts=[[13816, 5316, 831, 49],
[91, 4, 1, 19961], [2, 0, 0, 19929]]
i=14400, epochs=15.36, elapsed=0.16, elbo=-629.96
i=14400, accuracy=0.2427, counts=[[14558, 4969, 475, 10], [108, 4, 1,
19944], [1, 0, 0, 19930]]
i=14500, epochs=15.47, elapsed=0.16, elbo=-631.89
i=14500, accuracy=0.2529666666666667, counts=[[15172, 4434, 401, 5], [133,
6, 1, 19917], [4, 0, 0, 19927]]
i=14600, epochs=15.57, elapsed=0.16, elbo=-635.68
i=14600, accuracy=0.2530833333333333, counts=[[15183, 4415, 405, 9], [113,
2, 2, 19940], [1, 0, 0, 19930]]
i=14700, epochs=15.68, elapsed=0.16, elbo=-632.97
i=14700, accuracy=0.25226666666666664, counts=[[15132, 4456, 409, 15],
[101, 4, 1, 19951], [0, 0, 0, 19931]]
i=14800, epochs=15.79, elapsed=0.16, elbo=-629.58
i=14800, accuracy=0.25221666666666664, counts=[[15129, 4437, 425, 21],
[102, 4, 0, 19951], [1, 0, 0, 19930]]
i=14900, epochs=15.89, elapsed=0.16, elbo=-632.61
i=14900, accuracy=0.25875, counts=[[15522, 4154, 325, 11], [101, 3, 3,
19950], [0, 0, 0, 19931]]
i=15000, epochs=16.00, elapsed=0.16, elbo=-633.37
i=15000, accuracy=0.25695, counts=[[15414, 4249, 339, 10], [111, 3, 1,
19942], [0, 0, 0, 19931]]
i=15100, epochs=16.11, elapsed=0.17, elbo=-652.45
i=15100, accuracy=0.2536, counts=[[15211, 4412, 374, 15], [95, 5, 0,
19957], [2, 0, 0, 19929]]
i=15200, epochs=16.21, elapsed=0.17, elbo=-638.37
i=15200, accuracy=0.2586833333333333, counts=[[15519, 4124, 360, 9], [120,
2, 3, 19932], [0, 0, 0, 19931]]
i=15300, epochs=16.32, elapsed=0.17, elbo=-634.06
i=15300, accuracy=0.25833333333333336, counts=[[15498, 4176, 331, 7],
[116, 2, 0, 19939], [1, 0, 0, 19930]]
i=15400, epochs=16.43, elapsed=0.17, elbo=-643.02
i=15400, accuracy=0.25498333333333334, counts=[[15296, 4298, 400, 18],
[92, 3, 2, 19960], [1, 0, 0, 19930]]
i=15500, epochs=16.53, elapsed=0.17, elbo=-632.93
i=15500, accuracy=0.25211666666666666, counts=[[15124, 4402, 459, 27],
[99, 3, 3, 19952], [0, 0, 0, 19931]]
i=15600, epochs=16.64, elapsed=0.17, elbo=-637.37
i=15600, accuracy=0.25206666666666666, counts=[[15121, 4466, 403, 22],
[99, 3, 3, 19952], [3, 0, 0, 19928]]
i=15700, epochs=16.75, elapsed=0.17, elbo=-638.92
i=15700, accuracy=0.2639166666666667, counts=[[15831, 3865, 309, 7], [121,
4, 2, 19930], [1, 0, 0, 19930]]
i=15800, epochs=16.85, elapsed=0.17, elbo=-635.42
i=15800, accuracy=0.2516833333333333, counts=[[15097, 4401, 495, 19],
[100, 4, 3, 19950], [1, 0, 0, 19930]]
i=15900, epochs=16.96, elapsed=0.17, elbo=-643.07
i=15900, accuracy=0.24353333333333332, counts=[[14608, 4699, 662, 43],
[91, 2, 1, 19963], [0, 0, 2, 19929]]
i=16000, epochs=17.07, elapsed=0.18, elbo=-619.81
i=16000, accuracy=0.25033333333333335, counts=[[15018, 4509, 463, 22],
[107, 2, 2, 19946], [0, 0, 0, 19931]]
i=16100, epochs=17.17, elapsed=0.18, elbo=-626.70
i=16100, accuracy=0.2500833333333333, counts=[[15001, 4484, 493, 34], [90,
4, 3, 19960], [0, 0, 0, 19931]]
i=16200, epochs=17.28, elapsed=0.18, elbo=-654.04
i=16200, accuracy=0.24733333333333332, counts=[[14839, 4551, 584, 38],
[102, 1, 1, 19953], [0, 0, 0, 19931]]
i=16300, epochs=17.39, elapsed=0.18, elbo=-612.12
i=16300, accuracy=0.25635, counts=[[15378, 4197, 415, 22], [86, 2, 1,
19968], [0, 0, 1, 19930]]
i=16400, epochs=17.49, elapsed=0.18, elbo=-632.14
i=16400, accuracy=0.2565, counts=[[15386, 4295, 322, 9], [105, 2, 1,
19949], [1, 0, 2, 19928]]
i=16500, epochs=17.60, elapsed=0.18, elbo=-647.29
i=16500, accuracy=0.27091666666666664, counts=[[16251, 3613, 146, 2],
[112, 4, 2, 19939], [0, 0, 0, 19931]]
i=16600, epochs=17.71, elapsed=0.18, elbo=-635.98
i=16600, accuracy=0.27175, counts=[[16303, 3587, 122, 0], [93, 2, 3,
19959], [2, 0, 0, 19929]]
i=16700, epochs=17.81, elapsed=0.18, elbo=-628.25
i=16700, accuracy=0.25948333333333334, counts=[[15566, 4203, 240, 3], [87,
3, 2, 19965], [1, 0, 0, 19930]]
i=16800, epochs=17.92, elapsed=0.18, elbo=-634.81
i=16800, accuracy=0.2644, counts=[[15863, 3989, 159, 1], [114, 1, 0,
19942], [0, 0, 0, 19931]]
i=16900, epochs=18.03, elapsed=0.18, elbo=-631.89
i=16900, accuracy=0.2561333333333333, counts=[[15364, 4360, 283, 5], [84,
3, 2, 19968], [2, 0, 1, 19928]]
i=17000, epochs=18.13, elapsed=0.19, elbo=-633.59
i=17000, accuracy=0.25771666666666665, counts=[[15461, 4275, 272, 4], [96,
2, 0, 19959], [0, 0, 0, 19931]]
i=17100, epochs=18.24, elapsed=0.19, elbo=-635.94
i=17100, accuracy=0.2518, counts=[[15104, 4494, 404, 10], [86, 4, 1,
19966], [0, 0, 0, 19931]]
i=17200, epochs=18.35, elapsed=0.19, elbo=-631.00
i=17200, accuracy=0.2457, counts=[[14739, 4798, 463, 12], [75, 2, 1,
19979], [1, 0, 1, 19929]]
i=17300, epochs=18.45, elapsed=0.19, elbo=-636.46
i=17300, accuracy=0.2366, counts=[[14193, 5089, 691, 39], [60, 3, 1,
19993], [0, 0, 0, 19931]]
i=17400, epochs=18.56, elapsed=0.19, elbo=-624.47
i=17400, accuracy=0.23185, counts=[[13909, 5160, 898, 45], [69, 1, 1,
19986], [2, 0, 1, 19928]]
i=17500, epochs=18.67, elapsed=0.19, elbo=-628.96
i=17500, accuracy=0.23465, counts=[[14076, 5257, 643, 36], [82, 3, 4,
19968], [1, 0, 0, 19930]]
i=17600, epochs=18.77, elapsed=0.19, elbo=-643.30
i=17600, accuracy=0.2251, counts=[[13503, 5683, 789, 37], [57, 2, 2,
19996], [0, 0, 1, 19930]]
i=17700, epochs=18.88, elapsed=0.19, elbo=-646.70
i=17700, accuracy=0.22523333333333334, counts=[[13512, 5828, 654, 18],
[56, 2, 2, 19997], [0, 0, 0, 19931]]
i=17800, epochs=18.99, elapsed=0.19, elbo=-651.97
i=17800, accuracy=0.21831666666666666, counts=[[13097, 6032, 842, 41],
[52, 2, 3, 20000], [1, 0, 0, 19930]]
i=17900, epochs=19.09, elapsed=0.20, elbo=-631.12
i=17900, accuracy=0.22001666666666667, counts=[[13200, 5781, 975, 56],
[51, 1, 2, 20003], [2, 0, 0, 19929]]
i=18000, epochs=19.20, elapsed=0.20, elbo=-638.92
i=18000, accuracy=0.21548333333333333, counts=[[12926, 5928, 1070, 88],
[50, 2, 2, 20003], [1, 1, 1, 19928]]
i=18100, epochs=19.31, elapsed=0.20, elbo=-622.15
i=18100, accuracy=0.2287, counts=[[13717, 5600, 678, 17], [49, 5, 0,
20003], [0, 0, 0, 19931]]
i=18200, epochs=19.41, elapsed=0.20, elbo=-634.96
i=18200, accuracy=0.22101666666666667, counts=[[13261, 5806, 912, 33],
[51, 0, 0, 20006], [0, 0, 0, 19931]]
i=18300, epochs=19.52, elapsed=0.20, elbo=-632.12
i=18300, accuracy=0.21796666666666667, counts=[[13076, 6031, 858, 47],
[52, 2, 3, 20000], [0, 0, 0, 19931]]
i=18400, epochs=19.63, elapsed=0.20, elbo=-634.22
i=18400, accuracy=0.22178333333333333, counts=[[13304, 5879, 799, 30],
[44, 3, 0, 20010], [0, 0, 0, 19931]]
i=18500, epochs=19.73, elapsed=0.20, elbo=-633.45
i=18500, accuracy=0.22653333333333334, counts=[[13589, 5714, 684, 25],
[62, 3, 1, 19991], [1, 0, 0, 19930]]
i=18600, epochs=19.84, elapsed=0.20, elbo=-627.87
i=18600, accuracy=0.23551666666666668, counts=[[14127, 5351, 521, 13],
[69, 3, 1, 19984], [0, 0, 1, 19930]]
i=18700, epochs=19.95, elapsed=0.20, elbo=-642.75
i=18700, accuracy=0.23261666666666667, counts=[[13957, 5516, 528, 11],
[65, 0, 0, 19992], [0, 0, 0, 19931]]
i=18800, epochs=20.05, elapsed=0.21, elbo=-616.48
i=18800, accuracy=0.24001666666666666, counts=[[14399, 5221, 384, 8], [61,
2, 3, 19991], [0, 0, 0, 19931]]
i=18900, epochs=20.16, elapsed=0.21, elbo=-628.91
i=18900, accuracy=0.24653333333333333, counts=[[14789, 4890, 326, 7], [52,
3, 2, 20000], [0, 0, 0, 19931]]
i=19000, epochs=20.27, elapsed=0.21, elbo=-628.62
i=19000, accuracy=0.2504, counts=[[15022, 4701, 286, 3], [58, 2, 4,
19993], [1, 0, 0, 19930]]
i=19100, epochs=20.37, elapsed=0.21, elbo=-643.32
i=19100, accuracy=0.26245, counts=[[15747, 4110, 153, 2], [83, 0, 2,
19972], [1, 0, 0, 19930]]
i=19200, epochs=20.48, elapsed=0.21, elbo=-644.37
i=19200, accuracy=0.26156666666666667, counts=[[15687, 4147, 176, 2], [71,
7, 1, 19978], [1, 0, 0, 19930]]
i=19300, epochs=20.59, elapsed=0.21, elbo=-631.69
i=19300, accuracy=0.27063333333333334, counts=[[16236, 3696, 80, 0], [92,
2, 3, 19960], [1, 0, 0, 19930]]
i=19400, epochs=20.69, elapsed=0.21, elbo=-634.84
i=19400, accuracy=0.2748, counts=[[16479, 3474, 59, 0], [90, 9, 0, 19958],
[1, 0, 0, 19930]]
i=19500, epochs=20.80, elapsed=0.21, elbo=-646.46
i=19500, accuracy=0.27595, counts=[[16556, 3392, 64, 0], [83, 1, 1,
19972], [0, 0, 0, 19931]]
i=19600, epochs=20.91, elapsed=0.21, elbo=-641.42
i=19600, accuracy=0.27048333333333335, counts=[[16224, 3720, 68, 0], [94,
5, 3, 19955], [1, 0, 0, 19930]]
i=19700, epochs=21.01, elapsed=0.22, elbo=-639.21
i=19700, accuracy=0.27363333333333334, counts=[[16412, 3526, 74, 0], [65,
6, 1, 19985], [0, 0, 0, 19931]]
i=19800, epochs=21.12, elapsed=0.22, elbo=-640.36
i=19800, accuracy=0.27525, counts=[[16510, 3424, 78, 0], [87, 5, 1,
19964], [1, 0, 0, 19930]]
i=19900, epochs=21.23, elapsed=0.22, elbo=-627.66
i=19900, accuracy=0.27631666666666665, counts=[[16574, 3378, 60, 0], [78,
5, 1, 19973], [0, 0, 0, 19931]]
i=20000, epochs=21.33, elapsed=0.22, elbo=-630.32
i=20000, accuracy=0.26921666666666666, counts=[[16150, 3750, 112, 0], [64,
3, 1, 19989], [0, 0, 0, 19931]]
i=20100, epochs=21.44, elapsed=0.22, elbo=-630.40
i=20100, accuracy=0.27165, counts=[[16297, 3631, 84, 0], [70, 2, 4,
19981], [0, 0, 0, 19931]]
i=20200, epochs=21.55, elapsed=0.22, elbo=-625.16
i=20200, accuracy=0.2749333333333333, counts=[[16491, 3452, 69, 0], [74,
4, 1, 19978], [0, 0, 1, 19930]]
i=20300, epochs=21.65, elapsed=0.22, elbo=-643.68
i=20300, accuracy=0.2677333333333333, counts=[[16060, 3847, 105, 0], [60,
4, 1, 19992], [0, 0, 0, 19931]]
i=20400, epochs=21.76, elapsed=0.22, elbo=-640.27
i=20400, accuracy=0.2615166666666667, counts=[[15684, 4191, 137, 0], [61,
7, 7, 19982], [0, 0, 0, 19931]]
i=20500, epochs=21.87, elapsed=0.22, elbo=-631.60
i=20500, accuracy=0.25743333333333335, counts=[[15444, 4375, 192, 1], [58,
2, 2, 19995], [0, 0, 0, 19931]]
i=20600, epochs=21.97, elapsed=0.23, elbo=-632.31
i=20600, accuracy=0.2660166666666667, counts=[[15957, 3972, 82, 1], [56,
4, 5, 19992], [0, 0, 0, 19931]]
i=20700, epochs=22.08, elapsed=0.23, elbo=-643.56
i=20700, accuracy=0.27126666666666666, counts=[[16276, 3679, 57, 0], [67,
0, 0, 19990], [0, 0, 0, 19931]]
i=20800, epochs=22.19, elapsed=0.23, elbo=-638.55
i=20800, accuracy=0.27765, counts=[[16658, 3307, 47, 0], [69, 1, 1,
19986], [0, 0, 0, 19931]]
i=20900, epochs=22.29, elapsed=0.23, elbo=-618.23
i=20900, accuracy=0.27225, counts=[[16331, 3609, 72, 0], [57, 4, 4,
19992], [0, 0, 0, 19931]]
i=21000, epochs=22.40, elapsed=0.23, elbo=-632.58
i=21000, accuracy=0.2667333333333333, counts=[[16002, 3913, 97, 0], [48,
2, 4, 20003], [0, 0, 0, 19931]]
i=21100, epochs=22.51, elapsed=0.23, elbo=-646.97
i=21100, accuracy=0.25971666666666665, counts=[[15578, 4287, 146, 1], [59,
5, 2, 19991], [1, 0, 0, 19930]]
i=21200, epochs=22.61, elapsed=0.23, elbo=-620.29
i=21200, accuracy=0.25683333333333336, counts=[[15404, 4383, 224, 1], [60,
6, 4, 19987], [0, 0, 0, 19931]]
i=21300, epochs=22.72, elapsed=0.23, elbo=-645.11
i=21300, accuracy=0.2542, counts=[[15251, 4552, 207, 2], [39, 1, 5,
20012], [2, 0, 0, 19929]]
i=21400, epochs=22.83, elapsed=0.23, elbo=-636.19
i=21400, accuracy=0.26113333333333333, counts=[[15666, 4155, 191, 0], [56,
1, 3, 19997], [0, 0, 1, 19930]]
i=21500, epochs=22.93, elapsed=0.24, elbo=-632.57
i=21500, accuracy=0.2619666666666667, counts=[[15716, 4112, 182, 2], [61,
2, 1, 19993], [0, 0, 0, 19931]]
i=21600, epochs=23.04, elapsed=0.24, elbo=-623.33
i=21600, accuracy=0.26356666666666667, counts=[[15804, 3992, 212, 4], [57,
10, 1, 19989], [0, 0, 0, 19931]]
i=21700, epochs=23.15, elapsed=0.24, elbo=-625.43
i=21700, accuracy=0.2639, counts=[[15832, 3997, 181, 2], [47, 2, 5,
20003], [0, 0, 0, 19931]]
i=21800, epochs=23.25, elapsed=0.24, elbo=-624.74
i=21800, accuracy=0.2666, counts=[[15990, 3841, 180, 1], [67, 6, 1,
19983], [0, 0, 0, 19931]]
i=21900, epochs=23.36, elapsed=0.24, elbo=-631.57
i=21900, accuracy=0.2558166666666667, counts=[[15346, 4339, 319, 8], [59,
3, 3, 19992], [0, 0, 0, 19931]]
i=22000, epochs=23.47, elapsed=0.24, elbo=-634.05
i=22000, accuracy=0.2533, counts=[[15197, 4395, 408, 12], [55, 1, 5,
19996], [0, 0, 0, 19931]]
i=22100, epochs=23.57, elapsed=0.24, elbo=-636.52
i=22100, accuracy=0.2504166666666667, counts=[[15024, 4547, 431, 10], [61,
1, 2, 19993], [0, 0, 0, 19931]]
i=22200, epochs=23.68, elapsed=0.24, elbo=-639.59
i=22200, accuracy=0.24643333333333334, counts=[[14782, 4669, 544, 17],
[42, 3, 2, 20010], [0, 0, 1, 19930]]
i=22300, epochs=23.79, elapsed=0.24, elbo=-643.14
i=22300, accuracy=0.25353333333333333, counts=[[15210, 4480, 318, 4], [59,
2, 2, 19994], [0, 0, 0, 19931]]
i=22400, epochs=23.89, elapsed=0.25, elbo=-642.34
i=22400, accuracy=0.25161666666666666, counts=[[15094, 4520, 389, 9], [61,
3, 2, 19991], [0, 0, 0, 19931]]
i=22500, epochs=24.00, elapsed=0.25, elbo=-630.77
i=22500, accuracy=0.25301666666666667, counts=[[15177, 4444, 388, 3], [48,
4, 2, 20003], [0, 0, 0, 19931]]
i=22600, epochs=24.11, elapsed=0.25, elbo=-635.78
i=22600, accuracy=0.25476666666666664, counts=[[15282, 4442, 284, 4], [47,
4, 1, 20005], [1, 0, 0, 19930]]
i=22700, epochs=24.21, elapsed=0.25, elbo=-627.92
i=22700, accuracy=0.2547, counts=[[15280, 4436, 296, 0], [50, 2, 2,
20003], [0, 0, 0, 19931]]
i=22800, epochs=24.32, elapsed=0.25, elbo=-632.34
i=22800, accuracy=0.2573166666666667, counts=[[15435, 4331, 242, 4], [48,
4, 2, 20003], [1, 0, 0, 19930]]
i=22900, epochs=24.43, elapsed=0.25, elbo=-643.10
i=22900, accuracy=0.24558333333333332, counts=[[14733, 4744, 509, 26],
[58, 2, 4, 19993], [2, 0, 0, 19929]]
i=23000, epochs=24.53, elapsed=0.25, elbo=-632.29
i=23000, accuracy=0.25161666666666666, counts=[[15094, 4524, 389, 5], [44,
3, 3, 20007], [0, 0, 0, 19931]]
i=23100, epochs=24.64, elapsed=0.25, elbo=-642.42
i=23100, accuracy=0.23158333333333334, counts=[[13892, 5120, 911, 89],
[40, 3, 2, 20012], [0, 0, 0, 19931]]
i=23200, epochs=24.75, elapsed=0.25, elbo=-625.71
i=23200, accuracy=0.22795, counts=[[13676, 5306, 956, 74], [42, 1, 5,
20009], [0, 0, 0, 19931]]
i=23300, epochs=24.85, elapsed=0.26, elbo=-647.60
i=23300, accuracy=0.22003333333333333, counts=[[13202, 5225, 1327, 258],
[43, 0, 3, 20011], [0, 0, 0, 19931]]
i=23400, epochs=24.96, elapsed=0.26, elbo=-643.79
i=23400, accuracy=0.23316666666666666, counts=[[13985, 5073, 871, 83],
[44, 4, 5, 20004], [0, 0, 1, 19930]]
i=23500, epochs=25.07, elapsed=0.26, elbo=-638.00
i=23500, accuracy=0.23253333333333334, counts=[[13950, 5211, 783, 68],
[37, 2, 1, 20017], [0, 0, 0, 19931]]
i=23600, epochs=25.17, elapsed=0.26, elbo=-631.54
i=23600, accuracy=0.24135, counts=[[14479, 4788, 700, 45], [46, 2, 3,
20006], [0, 0, 0, 19931]]
i=23700, epochs=25.28, elapsed=0.26, elbo=-646.97
i=23700, accuracy=0.24601666666666666, counts=[[14759, 4565, 645, 43],
[45, 2, 3, 20007], [2, 0, 0, 19929]]
i=23800, epochs=25.39, elapsed=0.26, elbo=-635.45
i=23800, accuracy=0.2508166666666667, counts=[[15045, 4505, 445, 17], [52,
4, 1, 20000], [2, 0, 0, 19929]]
i=23900, epochs=25.49, elapsed=0.26, elbo=-632.98
i=23900, accuracy=0.25453333333333333, counts=[[15270, 4348, 388, 6], [54,
2, 0, 20001], [1, 0, 0, 19930]]
i=24000, epochs=25.60, elapsed=0.26, elbo=-632.16
i=24000, accuracy=0.2611833333333333, counts=[[15670, 3962, 363, 17], [47,
1, 0, 20009], [0, 1, 0, 19930]]
i=24100, epochs=25.71, elapsed=0.26, elbo=-644.85
i=24100, accuracy=0.25775, counts=[[15461, 4164, 371, 16], [52, 4, 1,
20000], [0, 0, 0, 19931]]
i=24200, epochs=25.81, elapsed=0.26, elbo=-630.80
i=24200, accuracy=0.25355, counts=[[15212, 4316, 469, 15], [44, 1, 3,
20009], [0, 0, 0, 19931]]
i=24300, epochs=25.92, elapsed=0.27, elbo=-637.52
i=24300, accuracy=0.2610166666666667, counts=[[15659, 4086, 261, 6], [49,
2, 2, 20004], [0, 0, 0, 19931]]
i=24400, epochs=26.03, elapsed=0.27, elbo=-626.44
i=24400, accuracy=0.2681, counts=[[16086, 3731, 193, 2], [38, 0, 1,
20018], [0, 0, 0, 19931]]
i=24500, epochs=26.13, elapsed=0.27, elbo=-641.01
i=24500, accuracy=0.2669666666666667, counts=[[16015, 3794, 202, 1], [55,
3, 2, 19997], [0, 0, 0, 19931]]
i=24600, epochs=26.24, elapsed=0.27, elbo=-637.78
i=24600, accuracy=0.26448333333333335, counts=[[15865, 3943, 201, 3], [54,
4, 3, 19996], [0, 0, 0, 19931]]
i=24700, epochs=26.35, elapsed=0.27, elbo=-648.84
i=24700, accuracy=0.26395, counts=[[15837, 3889, 281, 5], [48, 0, 1,
20008], [0, 0, 0, 19931]]
i=24800, epochs=26.45, elapsed=0.27, elbo=-643.06
i=24800, accuracy=0.26256666666666667, counts=[[15754, 3940, 311, 7], [41,
0, 2, 20014], [0, 0, 0, 19931]]
i=24900, epochs=26.56, elapsed=0.27, elbo=-636.86
i=24900, accuracy=0.2727, counts=[[16359, 3498, 152, 3], [55, 3, 1,
19998], [0, 0, 0, 19931]]
i=25000, epochs=26.67, elapsed=0.27, elbo=-639.07
i=25000, accuracy=0.2745, counts=[[16470, 3384, 156, 2], [59, 0, 2,
19996], [0, 0, 0, 19931]]
i=25100, epochs=26.77, elapsed=0.28, elbo=-649.28
i=25100, accuracy=0.2815, counts=[[16888, 3030, 94, 0], [57, 2, 0, 19998],
[0, 0, 0, 19931]]
i=25200, epochs=26.88, elapsed=0.28, elbo=-623.21
i=25200, accuracy=0.27903333333333336, counts=[[16741, 3155, 114, 2], [56,
1, 4, 19996], [0, 0, 0, 19931]]
i=25300, epochs=26.99, elapsed=0.28, elbo=-652.85
i=25300, accuracy=0.26748333333333335, counts=[[16047, 3772, 191, 2], [39,
2, 1, 20015], [0, 0, 0, 19931]]
i=25400, epochs=27.09, elapsed=0.28, elbo=-646.21
i=25400, accuracy=0.2641, counts=[[15845, 3924, 241, 2], [39, 1, 3,
20014], [1, 0, 0, 19930]]
i=25500, epochs=27.20, elapsed=0.28, elbo=-646.33
i=25500, accuracy=0.2703333333333333, counts=[[16219, 3635, 157, 1], [50,
1, 3, 20003], [0, 0, 0, 19931]]
i=25600, epochs=27.31, elapsed=0.28, elbo=-640.57
i=25600, accuracy=0.27098333333333335, counts=[[16258, 3610, 144, 0], [46,
0, 1, 20010], [0, 0, 1, 19930]]
i=25700, epochs=27.41, elapsed=0.28, elbo=-649.23
i=25700, accuracy=0.2766, counts=[[16595, 3318, 99, 0], [53, 1, 4, 19999],
[0, 0, 0, 19931]]
i=25800, epochs=27.52, elapsed=0.28, elbo=-635.80
i=25800, accuracy=0.28108333333333335, counts=[[16865, 3074, 73, 0], [49,
0, 2, 20006], [1, 0, 0, 19930]]
i=25900, epochs=27.63, elapsed=0.28, elbo=-631.90
i=25900, accuracy=0.28228333333333333, counts=[[16936, 3024, 52, 0], [59,
1, 1, 19996], [0, 0, 0, 19931]]
i=26000, epochs=27.73, elapsed=0.29, elbo=-633.73
i=26000, accuracy=0.29008333333333336, counts=[[17403, 2572, 37, 0], [61,
2, 2, 19992], [0, 0, 0, 19931]]
i=26100, epochs=27.84, elapsed=0.29, elbo=-630.84
i=26100, accuracy=0.2907166666666667, counts=[[17440, 2549, 23, 0], [48,
3, 0, 20006], [1, 0, 0, 19930]]
i=26200, epochs=27.95, elapsed=0.29, elbo=-658.76
i=26200, accuracy=0.29385, counts=[[17628, 2361, 23, 0], [56, 3, 0,
19998], [0, 0, 0, 19931]]
i=26300, epochs=28.05, elapsed=0.29, elbo=-639.56
i=26300, accuracy=0.29725, counts=[[17833, 2155, 24, 0], [62, 2, 1,
19992], [0, 0, 0, 19931]]
i=26400, epochs=28.16, elapsed=0.29, elbo=-643.51
i=26400, accuracy=0.3006666666666667, counts=[[18039, 1968, 5, 0], [44, 1,
0, 20012], [1, 0, 0, 19930]]
i=26500, epochs=28.27, elapsed=0.29, elbo=-628.47
i=26500, accuracy=0.3048166666666667, counts=[[18285, 1721, 6, 0], [52, 4,
0, 20001], [0, 0, 0, 19931]]
i=26600, epochs=28.37, elapsed=0.29, elbo=-647.63
i=26600, accuracy=0.30791666666666667, counts=[[18473, 1533, 6, 0], [55,
2, 1, 19999], [0, 0, 0, 19931]]
i=26700, epochs=28.48, elapsed=0.29, elbo=-633.17
i=26700, accuracy=0.3063, counts=[[18377, 1626, 9, 0], [60, 1, 1, 19995],
[0, 0, 0, 19931]]
i=26800, epochs=28.59, elapsed=0.29, elbo=-646.14
i=26800, accuracy=0.30651666666666666, counts=[[18388, 1615, 9, 0], [51,
3, 2, 20001], [0, 0, 0, 19931]]
i=26900, epochs=28.69, elapsed=0.29, elbo=-635.92
i=26900, accuracy=0.3080333333333333, counts=[[18479, 1527, 6, 0], [63, 3,
0, 19991], [0, 0, 0, 19931]]
i=27000, epochs=28.80, elapsed=0.30, elbo=-642.08
i=27000, accuracy=0.3087666666666667, counts=[[18522, 1484, 6, 0], [79, 4,
0, 19974], [0, 0, 0, 19931]]
i=27100, epochs=28.91, elapsed=0.30, elbo=-634.92
i=27100, accuracy=0.30656666666666665, counts=[[18393, 1617, 2, 0], [53,
1, 3, 20000], [0, 0, 0, 19931]]
i=27200, epochs=29.01, elapsed=0.30, elbo=-640.05
i=27200, accuracy=0.2929833333333333, counts=[[17577, 2411, 23, 1], [48,
2, 3, 20004], [0, 0, 0, 19931]]
i=27300, epochs=29.12, elapsed=0.30, elbo=-626.64
i=27300, accuracy=0.28631666666666666, counts=[[17177, 2781, 54, 0], [46,
2, 1, 20008], [0, 0, 0, 19931]]
i=27400, epochs=29.23, elapsed=0.30, elbo=-638.04
i=27400, accuracy=0.28781666666666667, counts=[[17267, 2710, 35, 0], [43,
2, 1, 20011], [0, 0, 0, 19931]]
i=27500, epochs=29.33, elapsed=0.30, elbo=-634.01
i=27500, accuracy=0.28941666666666666, counts=[[17364, 2615, 33, 0], [48,
1, 2, 20006], [1, 0, 0, 19930]]
i=27600, epochs=29.44, elapsed=0.30, elbo=-642.14
i=27600, accuracy=0.2947166666666667, counts=[[17680, 2298, 33, 1], [56,
3, 2, 19996], [0, 0, 0, 19931]]
i=27700, epochs=29.55, elapsed=0.30, elbo=-655.00
i=27700, accuracy=0.29496666666666665, counts=[[17698, 2295, 19, 0], [46,
0, 1, 20010], [0, 0, 0, 19931]]
i=27800, epochs=29.65, elapsed=0.30, elbo=-627.91
i=27800, accuracy=0.29795, counts=[[17875, 2124, 13, 0], [41, 2, 0,
20014], [0, 0, 0, 19931]]
i=27900, epochs=29.76, elapsed=0.31, elbo=-629.66
i=27900, accuracy=0.30651666666666666, counts=[[18389, 1616, 7, 0], [57,
2, 1, 19997], [1, 0, 0, 19930]]
i=28000, epochs=29.87, elapsed=0.31, elbo=-649.99
i=28000, accuracy=0.3070333333333333, counts=[[18418, 1585, 9, 0], [57, 4,
3, 19993], [0, 0, 0, 19931]]
i=28100, epochs=29.97, elapsed=0.31, elbo=-643.33
i=28100, accuracy=0.2956, counts=[[17734, 2255, 23, 0], [38, 2, 2, 20015],
[0, 0, 0, 19931]]
i=28200, epochs=30.08, elapsed=0.31, elbo=-630.60
i=28200, accuracy=0.299, counts=[[17937, 2047, 28, 0], [37, 3, 1, 20016],
[0, 0, 0, 19931]]
i=28300, epochs=30.19, elapsed=0.31, elbo=-644.18
i=28300, accuracy=0.30483333333333335, counts=[[18288, 1705, 19, 0], [36,
2, 1, 20018], [0, 0, 0, 19931]]
i=28400, epochs=30.29, elapsed=0.31, elbo=-648.34
i=28400, accuracy=0.31103333333333333, counts=[[18659, 1345, 8, 0], [41,
3, 0, 20013], [0, 0, 0, 19931]]
i=28500, epochs=30.40, elapsed=0.31, elbo=-630.37
i=28500, accuracy=0.3164, counts=[[18981, 1029, 2, 0], [65, 3, 1, 19988],
[1, 0, 0, 19930]]
i=28600, epochs=30.51, elapsed=0.31, elbo=-631.24
i=28600, accuracy=0.3162, counts=[[18972, 1038, 2, 0], [48, 0, 1, 20008],
[1, 0, 0, 19930]]
i=28700, epochs=30.61, elapsed=0.31, elbo=-632.17
i=28700, accuracy=0.3155, counts=[[18928, 1082, 2, 0], [56, 2, 4, 19995],
[0, 0, 0, 19931]]
i=28800, epochs=30.72, elapsed=0.32, elbo=-640.05
i=28800, accuracy=0.31471666666666664, counts=[[18881, 1129, 2, 0], [69,
2, 3, 19983], [0, 0, 0, 19931]]
i=28900, epochs=30.83, elapsed=0.32, elbo=-642.75
i=28900, accuracy=0.31883333333333336, counts=[[19130, 877, 5, 0], [64, 0,
1, 19992], [0, 0, 0, 19931]]
i=29000, epochs=30.93, elapsed=0.32, elbo=-637.94
i=29000, accuracy=0.3187333333333333, counts=[[19123, 887, 2, 0], [50, 1,
0, 20006], [0, 0, 0, 19931]]
i=29100, epochs=31.04, elapsed=0.32, elbo=-633.24
i=29100, accuracy=0.32048333333333334, counts=[[19227, 785, 0, 0], [71, 2,
0, 19984], [0, 0, 0, 19931]]
i=29200, epochs=31.15, elapsed=0.32, elbo=-642.02
i=29200, accuracy=0.31553333333333333, counts=[[18930, 1080, 2, 0], [51,
2, 2, 20002], [1, 0, 0, 19930]]
i=29300, epochs=31.25, elapsed=0.32, elbo=-628.74
i=29300, accuracy=0.3176, counts=[[19054, 957, 1, 0], [47, 2, 1, 20007],
[0, 0, 0, 19931]]
i=29400, epochs=31.36, elapsed=0.32, elbo=-643.77
i=29400, accuracy=0.31621666666666665, counts=[[18970, 1040, 2, 0], [57,
3, 0, 19997], [0, 0, 0, 19931]]
i=29500, epochs=31.47, elapsed=0.32, elbo=-635.53
i=29500, accuracy=0.3212333333333333, counts=[[19274, 737, 1, 0], [53, 0,
2, 20002], [0, 0, 0, 19931]]
i=29600, epochs=31.57, elapsed=0.32, elbo=-629.61
i=29600, accuracy=0.32176666666666665, counts=[[19304, 707, 1, 0], [65, 2,
1, 19989], [0, 0, 0, 19931]]
i=29700, epochs=31.68, elapsed=0.33, elbo=-648.29
i=29700, accuracy=0.32053333333333334, counts=[[19228, 784, 0, 0], [57, 4,
1, 19995], [0, 0, 0, 19931]]
i=29800, epochs=31.79, elapsed=0.33, elbo=-646.77
i=29800, accuracy=0.31971666666666665, counts=[[19179, 833, 0, 0], [57, 4,
0, 19996], [0, 0, 0, 19931]]
i=29900, epochs=31.89, elapsed=0.33, elbo=-641.06
i=29900, accuracy=0.32175, counts=[[19301, 711, 0, 0], [65, 4, 1, 19987],
[0, 0, 0, 19931]]
i=30000, epochs=32.00, elapsed=0.33, elbo=-643.42
i=30000, accuracy=0.32221666666666665, counts=[[19331, 681, 0, 0], [86, 2,
0, 19969], [0, 0, 0, 19931]]
i=30100, epochs=32.11, elapsed=0.33, elbo=-635.58
i=30100, accuracy=0.3227333333333333, counts=[[19361, 651, 0, 0], [73, 3,
1, 19980], [0, 0, 0, 19931]]
i=30200, epochs=32.21, elapsed=0.33, elbo=-627.51
i=30200, accuracy=0.3227833333333333, counts=[[19364, 646, 2, 0], [64, 3,
1, 19989], [0, 0, 0, 19931]]
i=30300, epochs=32.32, elapsed=0.33, elbo=-637.57
i=30300, accuracy=0.3233, counts=[[19395, 616, 1, 0], [60, 3, 0, 19994],
[0, 0, 0, 19931]]
i=30400, epochs=32.43, elapsed=0.33, elbo=-638.97
i=30400, accuracy=0.3227333333333333, counts=[[19361, 651, 0, 0], [57, 3,
1, 19996], [0, 0, 0, 19931]]
i=30500, epochs=32.53, elapsed=0.33, elbo=-649.53
i=30500, accuracy=0.3245166666666667, counts=[[19470, 542, 0, 0], [78, 1,
0, 19978], [0, 0, 0, 19931]]
i=30600, epochs=32.64, elapsed=0.33, elbo=-656.32
i=30600, accuracy=0.32233333333333336, counts=[[19338, 674, 0, 0], [60, 2,
1, 19994], [0, 0, 0, 19931]]
i=30700, epochs=32.75, elapsed=0.34, elbo=-647.96
i=30700, accuracy=0.32276666666666665, counts=[[19365, 647, 0, 0], [51, 1,
2, 20003], [0, 0, 0, 19931]]
i=30800, epochs=32.85, elapsed=0.34, elbo=-639.84
i=30800, accuracy=0.32255, counts=[[19352, 660, 0, 0], [68, 1, 0, 19988],
[1, 0, 0, 19930]]
i=30900, epochs=32.96, elapsed=0.34, elbo=-626.33
i=30900, accuracy=0.32285, counts=[[19370, 642, 0, 0], [77, 1, 0, 19979],
[0, 0, 0, 19931]]
i=31000, epochs=33.07, elapsed=0.34, elbo=-633.46
i=31000, accuracy=0.3251, counts=[[19502, 510, 0, 0], [74, 4, 2, 19977],
[0, 0, 0, 19931]]
i=31100, epochs=33.17, elapsed=0.34, elbo=-643.60
i=31100, accuracy=0.32503333333333334, counts=[[19500, 511, 1, 0], [75, 2,
0, 19980], [0, 0, 0, 19931]]
i=31200, epochs=33.28, elapsed=0.34, elbo=-651.35
i=31200, accuracy=0.32698333333333335, counts=[[19617, 395, 0, 0], [80, 2,
1, 19974], [0, 0, 0, 19931]]
i=31300, epochs=33.39, elapsed=0.34, elbo=-641.97
i=31300, accuracy=0.32738333333333336, counts=[[19640, 372, 0, 0], [95, 3,
1, 19958], [0, 0, 0, 19931]]
i=31400, epochs=33.49, elapsed=0.34, elbo=-642.20
i=31400, accuracy=0.32743333333333335, counts=[[19644, 367, 1, 0], [64, 2,
1, 19990], [0, 0, 0, 19931]]
i=31500, epochs=33.60, elapsed=0.34, elbo=-631.46
i=31500, accuracy=0.3267, counts=[[19601, 411, 0, 0], [77, 1, 1, 19978],
[0, 0, 0, 19931]]
i=31600, epochs=33.71, elapsed=0.35, elbo=-642.61
i=31600, accuracy=0.3263666666666667, counts=[[19580, 432, 0, 0], [75, 2,
0, 19980], [0, 0, 0, 19931]]
i=31700, epochs=33.81, elapsed=0.35, elbo=-642.25
i=31700, accuracy=0.3255166666666667, counts=[[19530, 482, 0, 0], [52, 1,
1, 20003], [0, 0, 0, 19931]]
i=31800, epochs=33.92, elapsed=0.35, elbo=-640.49
i=31800, accuracy=0.32525, counts=[[19512, 499, 1, 0], [75, 3, 0, 19979],
[0, 0, 0, 19931]]
i=31900, epochs=34.03, elapsed=0.35, elbo=-645.67
i=31900, accuracy=0.3255166666666667, counts=[[19529, 483, 0, 0], [65, 2,
1, 19989], [0, 0, 0, 19931]]
i=32000, epochs=34.13, elapsed=0.35, elbo=-644.68
i=32000, accuracy=0.32661666666666667, counts=[[19597, 415, 0, 0], [76, 0,
2, 19979], [0, 0, 0, 19931]]
i=32100, epochs=34.24, elapsed=0.35, elbo=-648.79
i=32100, accuracy=0.3247833333333333, counts=[[19484, 528, 0, 0], [53, 3,
1, 20000], [0, 0, 0, 19931]]
i=32200, epochs=34.35, elapsed=0.35, elbo=-638.69
i=32200, accuracy=0.3253666666666667, counts=[[19521, 490, 1, 0], [66, 1,
1, 19989], [1, 0, 0, 19930]]
i=32300, epochs=34.45, elapsed=0.35, elbo=-628.16
i=32300, accuracy=0.3253333333333333, counts=[[19516, 496, 0, 0], [70, 4,
4, 19979], [1, 0, 0, 19930]]
i=32400, epochs=34.56, elapsed=0.35, elbo=-635.70
i=32400, accuracy=0.32513333333333333, counts=[[19506, 506, 0, 0], [60, 2,
1, 19994], [0, 0, 0, 19931]]
i=32500, epochs=34.67, elapsed=0.36, elbo=-634.20
i=32500, accuracy=0.3199666666666667, counts=[[19196, 812, 4, 0], [54, 2,
0, 20001], [0, 0, 0, 19931]]
i=32600, epochs=34.77, elapsed=0.36, elbo=-650.96
i=32600, accuracy=0.3173, counts=[[19036, 975, 1, 0], [59, 2, 0, 19996],
[0, 0, 0, 19931]]
i=32700, epochs=34.88, elapsed=0.36, elbo=-629.92
i=32700, accuracy=0.3179166666666667, counts=[[19075, 936, 1, 0], [51, 0,
0, 20006], [0, 0, 0, 19931]]
i=32800, epochs=34.99, elapsed=0.36, elbo=-625.32
i=32800, accuracy=0.31876666666666664, counts=[[19124, 888, 0, 0], [41, 2,
2, 20012], [0, 0, 0, 19931]]
i=32900, epochs=35.09, elapsed=0.36, elbo=-649.14
i=32900, accuracy=0.32, counts=[[19199, 811, 2, 0], [42, 1, 0, 20014], [0,
0, 0, 19931]]
i=33000, epochs=35.20, elapsed=0.36, elbo=-634.82
i=33000, accuracy=0.32356666666666667, counts=[[19413, 599, 0, 0], [58, 1,
0, 19998], [0, 0, 0, 19931]]
i=33100, epochs=35.31, elapsed=0.36, elbo=-639.27
i=33100, accuracy=0.32208333333333333, counts=[[19325, 687, 0, 0], [60, 0,
0, 19997], [0, 0, 0, 19931]]
i=33200, epochs=35.41, elapsed=0.36, elbo=-640.96
i=33200, accuracy=0.3237833333333333, counts=[[19425, 587, 0, 0], [50, 2,
0, 20005], [0, 0, 0, 19931]]
i=33300, epochs=35.52, elapsed=0.36, elbo=-651.25
i=33300, accuracy=0.32516666666666666, counts=[[19510, 502, 0, 0], [65, 0,
0, 19992], [0, 0, 0, 19931]]
i=33400, epochs=35.63, elapsed=0.37, elbo=-650.75
i=33400, accuracy=0.32558333333333334, counts=[[19532, 479, 1, 0], [58, 3,
0, 19996], [1, 0, 0, 19930]]
i=33500, epochs=35.73, elapsed=0.37, elbo=-639.87
i=33500, accuracy=0.32726666666666665, counts=[[19636, 376, 0, 0], [48, 0,
1, 20008], [0, 0, 0, 19931]]
i=33600, epochs=35.84, elapsed=0.37, elbo=-642.97
i=33600, accuracy=0.3255, counts=[[19530, 482, 0, 0], [54, 0, 0, 20003],
[1, 0, 0, 19930]]
i=33700, epochs=35.95, elapsed=0.37, elbo=-647.52
i=33700, accuracy=0.3243333333333333, counts=[[19460, 551, 1, 0], [43, 0,
0, 20014], [0, 0, 0, 19931]]
i=33800, epochs=36.05, elapsed=0.37, elbo=-649.25
i=33800, accuracy=0.32248333333333334, counts=[[19347, 664, 1, 0], [33, 2,
0, 20022], [0, 0, 0, 19931]]
i=33900, epochs=36.16, elapsed=0.37, elbo=-641.13
i=33900, accuracy=0.3233, counts=[[19398, 613, 1, 0], [42, 0, 0, 20015],
[0, 0, 0, 19931]]
i=34000, epochs=36.27, elapsed=0.37, elbo=-630.09
i=34000, accuracy=0.3254666666666667, counts=[[19528, 484, 0, 0], [45, 0,
2, 20010], [0, 0, 0, 19931]]
i=34100, epochs=36.37, elapsed=0.37, elbo=-641.50
i=34100, accuracy=0.32526666666666665, counts=[[19516, 495, 1, 0], [48, 0,
0, 20009], [0, 0, 0, 19931]]
i=34200, epochs=36.48, elapsed=0.37, elbo=-642.03
i=34200, accuracy=0.3249166666666667, counts=[[19495, 517, 0, 0], [45, 0,
0, 20012], [0, 0, 0, 19931]]
i=34300, epochs=36.59, elapsed=0.38, elbo=-645.26
i=34300, accuracy=0.32158333333333333, counts=[[19295, 715, 2, 0], [37, 0,
0, 20020], [0, 0, 0, 19931]]
i=34400, epochs=36.69, elapsed=0.38, elbo=-639.94
i=34400, accuracy=0.3217333333333333, counts=[[19303, 708, 1, 0], [29, 1,
0, 20027], [0, 0, 0, 19931]]
i=34500, epochs=36.80, elapsed=0.38, elbo=-620.77
i=34500, accuracy=0.3251, counts=[[19503, 509, 0, 0], [37, 3, 1, 20016],
[0, 0, 0, 19931]]
i=34600, epochs=36.91, elapsed=0.38, elbo=-640.48
i=34600, accuracy=0.3238666666666667, counts=[[19430, 581, 1, 0], [35, 2,
0, 20020], [0, 0, 0, 19931]]
i=34700, epochs=37.01, elapsed=0.38, elbo=-639.58
i=34700, accuracy=0.32561666666666667, counts=[[19537, 475, 0, 0], [42, 0,
1, 20014], [0, 0, 0, 19931]]
i=34800, epochs=37.12, elapsed=0.38, elbo=-645.78
i=34800, accuracy=0.32365, counts=[[19418, 594, 0, 0], [24, 1, 0, 20032],
[0, 0, 0, 19931]]
i=34900, epochs=37.23, elapsed=0.38, elbo=-645.91
i=34900, accuracy=0.3232833333333333, counts=[[19396, 616, 0, 0], [35, 1,
0, 20021], [0, 0, 0, 19931]]
i=35000, epochs=37.33, elapsed=0.38, elbo=-643.04
i=35000, accuracy=0.32083333333333336, counts=[[19250, 761, 1, 0], [39, 0,
0, 20018], [0, 0, 0, 19931]]
i=35100, epochs=37.44, elapsed=0.38, elbo=-658.70
i=35100, accuracy=0.3199, counts=[[19194, 816, 2, 0], [42, 0, 1, 20014],
[0, 0, 0, 19931]]
i=35200, epochs=37.55, elapsed=0.39, elbo=-636.82
i=35200, accuracy=0.3183166666666667, counts=[[19098, 911, 3, 0], [35, 1,
0, 20021], [0, 0, 0, 19931]]
i=35300, epochs=37.65, elapsed=0.39, elbo=-642.01
i=35300, accuracy=0.3184166666666667, counts=[[19105, 906, 1, 0], [33, 0,
1, 20023], [0, 0, 0, 19931]]
i=35400, epochs=37.76, elapsed=0.39, elbo=-638.95
i=35400, accuracy=0.3222833333333333, counts=[[19336, 676, 0, 0], [41, 1,
0, 20015], [0, 0, 0, 19931]]
i=35500, epochs=37.87, elapsed=0.39, elbo=-637.37
i=35500, accuracy=0.32001666666666667, counts=[[19200, 812, 0, 0], [39, 1,
1, 20016], [0, 0, 0, 19931]]
i=35600, epochs=37.97, elapsed=0.39, elbo=-645.98
i=35600, accuracy=0.3194166666666667, counts=[[19165, 846, 1, 0], [38, 0,
0, 20019], [0, 0, 0, 19931]]
i=35700, epochs=38.08, elapsed=0.39, elbo=-651.00
i=35700, accuracy=0.31893333333333335, counts=[[19135, 876, 1, 0], [40, 1,
0, 20016], [0, 0, 0, 19931]]
i=35800, epochs=38.19, elapsed=0.39, elbo=-648.21
i=35800, accuracy=0.31758333333333333, counts=[[19052, 959, 1, 0], [41, 3,
1, 20012], [0, 0, 0, 19931]]
i=35900, epochs=38.29, elapsed=0.39, elbo=-625.44
i=35900, accuracy=0.31885, counts=[[19130, 880, 2, 0], [52, 1, 0, 20004],
[0, 0, 0, 19931]]
i=36000, epochs=38.40, elapsed=0.39, elbo=-636.04
i=36000, accuracy=0.3212833333333333, counts=[[19276, 736, 0, 0], [46, 1,
0, 20010], [0, 0, 0, 19931]]
i=36100, epochs=38.51, elapsed=0.39, elbo=-641.48
i=36100, accuracy=0.31916666666666665, counts=[[19149, 863, 0, 0], [45, 1,
1, 20010], [0, 0, 0, 19931]]
i=36200, epochs=38.61, elapsed=0.40, elbo=-639.10
i=36200, accuracy=0.316, counts=[[18960, 1049, 3, 0], [36, 0, 1, 20020],
[0, 0, 0, 19931]]
i=36300, epochs=38.72, elapsed=0.40, elbo=-653.89
i=36300, accuracy=0.31615, counts=[[18967, 1040, 5, 0], [37, 2, 0, 20018],
[1, 0, 0, 19930]]
i=36400, epochs=38.83, elapsed=0.40, elbo=-630.34
i=36400, accuracy=0.31848333333333334, counts=[[19109, 903, 0, 0], [57, 0,
0, 20000], [0, 0, 0, 19931]]
i=36500, epochs=38.93, elapsed=0.40, elbo=-635.60
i=36500, accuracy=0.31926666666666664, counts=[[19156, 855, 1, 0], [57, 0,
1, 19999], [0, 0, 0, 19931]]
i=36600, epochs=39.04, elapsed=0.40, elbo=-633.04
i=36600, accuracy=0.3168166666666667, counts=[[19009, 1002, 1, 0], [46, 0,
0, 20011], [0, 0, 0, 19931]]
i=36700, epochs=39.15, elapsed=0.40, elbo=-648.94
i=36700, accuracy=0.31801666666666667, counts=[[19080, 931, 1, 0], [46, 1,
2, 20008], [0, 0, 0, 19931]]
i=36800, epochs=39.25, elapsed=0.40, elbo=-640.54
i=36800, accuracy=0.3192333333333333, counts=[[19152, 859, 1, 0], [55, 2,
0, 20000], [0, 0, 0, 19931]]
i=36900, epochs=39.36, elapsed=0.40, elbo=-637.39
i=36900, accuracy=0.3216833333333333, counts=[[19301, 711, 0, 0], [53, 0,
0, 20004], [0, 0, 0, 19931]]
i=37000, epochs=39.47, elapsed=0.40, elbo=-642.99
i=37000, accuracy=0.32076666666666664, counts=[[19246, 764, 2, 0], [67, 0,
1, 19989], [0, 0, 0, 19931]]
i=37100, epochs=39.57, elapsed=0.41, elbo=-643.12
i=37100, accuracy=0.3223666666666667, counts=[[19342, 670, 0, 0], [61, 0,
0, 19996], [0, 0, 0, 19931]]
i=37200, epochs=39.68, elapsed=0.41, elbo=-642.23
i=37200, accuracy=0.3233333333333333, counts=[[19396, 616, 0, 0], [51, 4,
0, 20002], [0, 0, 0, 19931]]
i=37300, epochs=39.79, elapsed=0.41, elbo=-627.72
i=37300, accuracy=0.3197833333333333, counts=[[19186, 826, 0, 0], [47, 1,
0, 20009], [0, 0, 0, 19931]]
i=37400, epochs=39.89, elapsed=0.41, elbo=-635.96
i=37400, accuracy=0.31998333333333334, counts=[[19199, 812, 1, 0], [56, 0,
0, 20001], [0, 0, 0, 19931]]
i=37500, epochs=40.00, elapsed=0.41, elbo=-647.16
i=37500, accuracy=0.3193, counts=[[19156, 856, 0, 0], [48, 2, 0, 20007],
[1, 0, 0, 19930]]
i=37600, epochs=40.11, elapsed=0.41, elbo=-631.53
i=37600, accuracy=0.3165, counts=[[18989, 1019, 4, 0], [43, 1, 0, 20013],
[0, 0, 0, 19931]]
i=37700, epochs=40.21, elapsed=0.41, elbo=-650.86
i=37700, accuracy=0.31538333333333335, counts=[[18921, 1088, 3, 0], [37,
2, 0, 20018], [0, 0, 0, 19931]]
i=37800, epochs=40.32, elapsed=0.41, elbo=-641.33
i=37800, accuracy=0.31683333333333336, counts=[[19009, 1001, 2, 0], [41,
1, 0, 20015], [0, 0, 0, 19931]]
i=37900, epochs=40.43, elapsed=0.41, elbo=-644.82
i=37900, accuracy=0.31916666666666665, counts=[[19148, 863, 1, 0], [46, 2,
0, 20009], [1, 0, 0, 19930]]
i=38000, epochs=40.53, elapsed=0.42, elbo=-639.63
i=38000, accuracy=0.3185, counts=[[19109, 901, 2, 0], [39, 1, 0, 20017],
[0, 0, 0, 19931]]
i=38100, epochs=40.64, elapsed=0.42, elbo=-646.95
i=38100, accuracy=0.3198, counts=[[19188, 823, 1, 0], [34, 0, 1, 20022],
[0, 0, 0, 19931]]
i=38200, epochs=40.75, elapsed=0.42, elbo=-637.34
i=38200, accuracy=0.31951666666666667, counts=[[19171, 838, 3, 0], [36, 0,
0, 20021], [0, 0, 0, 19931]]
i=38300, epochs=40.85, elapsed=0.42, elbo=-639.44
i=38300, accuracy=0.3191833333333333, counts=[[19151, 861, 0, 0], [44, 0,
0, 20013], [0, 0, 0, 19931]]
i=38400, epochs=40.96, elapsed=0.42, elbo=-639.50
i=38400, accuracy=0.3187833333333333, counts=[[19126, 884, 2, 0], [32, 1,
0, 20024], [0, 0, 0, 19931]]
i=38500, epochs=41.07, elapsed=0.42, elbo=-646.87
i=38500, accuracy=0.3220166666666667, counts=[[19321, 691, 0, 0], [47, 0,
0, 20010], [0, 0, 0, 19931]]
i=38600, epochs=41.17, elapsed=0.42, elbo=-631.01
i=38600, accuracy=0.3208666666666667, counts=[[19252, 759, 1, 0], [50, 0,
0, 20007], [1, 0, 0, 19930]]
i=38700, epochs=41.28, elapsed=0.42, elbo=-652.03
i=38700, accuracy=0.3215166666666667, counts=[[19290, 720, 2, 0], [51, 1,
0, 20005], [0, 0, 0, 19931]]
i=38800, epochs=41.39, elapsed=0.42, elbo=-648.51
i=38800, accuracy=0.3211, counts=[[19266, 745, 1, 0], [39, 0, 0, 20018],
[1, 0, 0, 19930]]
i=38900, epochs=41.49, elapsed=0.43, elbo=-640.14
i=38900, accuracy=0.31961666666666666, counts=[[19176, 831, 5, 0], [46, 1,
0, 20010], [0, 0, 0, 19931]]
i=39000, epochs=41.60, elapsed=0.43, elbo=-653.06
i=39000, accuracy=0.32066666666666666, counts=[[19240, 772, 0, 0], [37, 0,
0, 20020], [0, 0, 0, 19931]]
i=39100, epochs=41.71, elapsed=0.43, elbo=-643.18
i=39100, accuracy=0.32066666666666666, counts=[[19239, 770, 3, 0], [42, 1,
0, 20014], [0, 0, 0, 19931]]
i=39200, epochs=41.81, elapsed=0.43, elbo=-649.61
i=39200, accuracy=0.32088333333333335, counts=[[19252, 758, 2, 0], [51, 1,
0, 20005], [0, 0, 0, 19931]]
i=39300, epochs=41.92, elapsed=0.43, elbo=-640.86
i=39300, accuracy=0.3218, counts=[[19307, 705, 0, 0], [37, 1, 0, 20019],
[0, 0, 0, 19931]]
i=39400, epochs=42.03, elapsed=0.43, elbo=-647.21
i=39400, accuracy=0.32405, counts=[[19443, 569, 0, 0], [39, 0, 0, 20018],
[0, 0, 0, 19931]]
i=39500, epochs=42.13, elapsed=0.43, elbo=-637.26
i=39500, accuracy=0.32293333333333335, counts=[[19376, 634, 2, 0], [41, 0,
0, 20016], [0, 0, 0, 19931]]
i=39600, epochs=42.24, elapsed=0.43, elbo=-646.94
i=39600, accuracy=0.32395, counts=[[19436, 576, 0, 0], [47, 1, 1, 20008],
[0, 0, 0, 19931]]
i=39700, epochs=42.35, elapsed=0.43, elbo=-638.31
i=39700, accuracy=0.32461666666666666, counts=[[19476, 536, 0, 0], [49, 1,
0, 20007], [0, 0, 0, 19931]]
i=39800, epochs=42.45, elapsed=0.44, elbo=-638.56
i=39800, accuracy=0.3256, counts=[[19534, 478, 0, 0], [45, 2, 0, 20010],
[0, 0, 0, 19931]]
i=39900, epochs=42.56, elapsed=0.44, elbo=-633.72
i=39900, accuracy=0.32363333333333333, counts=[[19417, 595, 0, 0], [54, 1,
1, 20001], [0, 0, 0, 19931]]
i=40000, epochs=42.67, elapsed=0.44, elbo=-641.45
i=40000, accuracy=0.32425, counts=[[19455, 557, 0, 0], [44, 0, 0, 20013],
[0, 0, 0, 19931]]
i=40100, epochs=42.77, elapsed=0.44, elbo=-640.18
i=40100, accuracy=0.3240166666666667, counts=[[19440, 571, 1, 0], [44, 1,
0, 20012], [0, 0, 0, 19931]]
i=40200, epochs=42.88, elapsed=0.44, elbo=-630.82
i=40200, accuracy=0.3254166666666667, counts=[[19523, 489, 0, 0], [48, 1,
0, 20008], [0, 0, 1, 19930]]
i=40300, epochs=42.99, elapsed=0.44, elbo=-642.28
i=40300, accuracy=0.32298333333333334, counts=[[19379, 632, 1, 0], [43, 0,
0, 20014], [1, 0, 0, 19930]]
i=40400, epochs=43.09, elapsed=0.44, elbo=-647.27
i=40400, accuracy=0.3215166666666667, counts=[[19291, 721, 0, 0], [33, 0,
0, 20024], [0, 0, 0, 19931]]
i=40500, epochs=43.20, elapsed=0.44, elbo=-646.54
i=40500, accuracy=0.32253333333333334, counts=[[19352, 659, 1, 0], [32, 0,
0, 20025], [0, 0, 0, 19931]]
i=40600, epochs=43.31, elapsed=0.45, elbo=-647.35
i=40600, accuracy=0.32015, counts=[[19209, 803, 0, 0], [36, 0, 1, 20020],
[0, 0, 0, 19931]]
i=

[Minju-Jung]

@martinjankowiak Could you kindly let me know what setting do I need to follow?
I added the option --z-pres-prior 0.01, but it looks the same as before.

[martinjankowiak]

refer to the bottom of the tutorial.

On Dec 19, 2017 9:26 AM, "Minju Jung" [e-mail] wrote:

@martinjankowiak https://github.com/martinjankowiak Could you kindly
let me know what setting do I need to follow?
I added the option --z-pres-prior 0.01, but it looks the same as before.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/uber/pyro/issues/637#issuecomment-352769328, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AVFMguRBs3JWTEMPA_4Y7iSFef6DylLPks5tB8eBgaJpZM4RG_HJ
.

[Minju-Jung]

@martinjankowiak I checked the tutorial as you recommended, but still I cannot solve the problem.

1. Inference is unlikely to recover correct object counts unless a small prior success probability for z_pres is used. In [1] this probability was annealed from a value close to one to 1e-5 (or less) during optimization, though we found that a fixed value of around 0.01 worked well with our implementation.
   --> I added the option --z-pres-prior 0.01.

2. We initialize the decoder network to generate mostly empty objects initially. (Using the --decoder-output-bias argument.) This encourages the guide to explore the use of objects to explain the input early in optimization. Without this each object is a mid-gray square which is heavily penalized by the likelihood, prompting the guide to turn most steps off.
   --> There is no exact setting for --decoder-output-bias. I set --decoder-output-bias to -1.

3. It is reported to be useful in practice to use a different learning rate for the baseline network. This is straight forward to implement in Pyro by tagging modules associated with the baseline network and passing multiple learning rates to the optimizer. (See the section on optimizers in part I of the SVI tutorial for more detail.) In [1] a learning rate of 1e-4 was used for the guide network, and a learning rate of 1e-3 was used for the baseline network. We found it necessary to use a larger learning rate for the baseline network in order to make progress on count accuracy at a similar rate to [1]. This difference is likely caused by Pyro setting up a slightly different baseline loss.
   --> These are the default setting of main.py

Could you kindly let me know the exact setting or options you used?

[martinjankowiak]

just above that you find

python main.py -n 200000 -blr 0.1 --z-pres-prior 0.01 --scale-prior-sd
0.2 --predict-net 200 --bl-predict-net 200
--decoder-output-use-sigmoid --decoder-output-bias -2 --seed 287710

On Dec 19, 2017 9:38 AM, "Minju Jung" [e-mail] wrote:

@martinjankowiak https://github.com/martinjankowiak I checked the
tutorial as you recommended, but still I cannot solve the problem.

1.

Inference is unlikely to recover correct object counts unless a small
prior success probability for z_pres is used. In [1] this probability was
annealed from a value close to one to 1e-5 (or less) during optimization,
though we found that a fixed value of around 0.01 worked well with our
implementation.
--> I added the option --z-pres-prior 0.01.
2.

We initialize the decoder network to generate mostly empty objects
initially. (Using the --decoder-output-bias argument.) This encourages the
guide to explore the use of objects to explain the input early in
optimization. Without this each object is a mid-gray square which is
heavily penalized by the likelihood, prompting the guide to turn most steps
off.
--> There is no exact setting for --decoder-output-bias. I set
--decoder-output-bias to -1.
3.

It is reported to be useful in practice to use a different learning
rate for the baseline network. This is straight forward to implement in
Pyro by tagging modules associated with the baseline network and passing
multiple learning rates to the optimizer. (See the section on optimizers in
part I of the SVI tutorial for more detail.) In [1] a learning rate of 1e-4
was used for the guide network, and a learning rate of 1e-3 was used for
the baseline network. We found it necessary to use a larger learning rate
for the baseline network in order to make progress on count accuracy at a
similar rate to [1]. This difference is likely caused by Pyro setting up a
slightly different baseline loss.
--> This are the default setting of main.py

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/uber/pyro/issues/637#issuecomment-352774125, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AVFMgodMibUbK7aDPy7remdms56dU9WCks5tB8pXgaJpZM4RG_HJ
.

[Minju-Jung]

@martinjankowiak Thank you very much!!
Now, I succeeded to reproduce the result based on your setting.
I think that this setting is better to be the default setting of main.py.