[01:37:06] i've written this telegram bot; https://github.com/proletesseract/bash-scripts/blob/master/nav-monitor.sh [01:38:09] it runs a local node, looks at the block 5 below the current tip and queries navexplorer and cryptoid to check some things about it like how many confirmations it has and what time it was minted to basically sanity check that the 3 nodes all agree on the latest blockchain. [01:39:11] if its successful it pings a success message to the telegram channel and if it finds some problem, it will ping some debug data and also @ tag me in the message so i know when something is wrong [01:39:38] i wrote it as a monitor tool for the mainnet that binance can potentially subscribe to for peace of mind that everything is functioning normally [01:40:09] if anyone wants to be added to the channel and added to the @ tag (or email list) for alerts on discrepencies, please just leave your telegram handle here or email address and i will add you to the channel and bot [01:40:18] it will only tag you if an error is found [01:40:44] sure add me [01:41:15] i the block hash mismatches, or the block is more than 7m30s older than the server time it will ping [01:41:45] i tried to do it on bestblock but there are always re-orgs causing the error to falsely report [01:42:31] so i chose 5 blocks deep and a factor of 3 tolerance level for the timestamp given the script is setup with some while loops to retry the api endpoints in case it can't get a response for some reason [01:42:36] its running with no errors [01:43:22] at this tolerance threshold [01:43:31] it runs every 10 minutes [01:43:52] so i guess we should always be alerted within ~18 minutes of some error at the latest [01:45:52] okay, ive added you and your @ tag [01:46:01] i will force it to create an error just to test the tagging is correct [01:47:18] okay, looks like it works [01:47:52] feel free to add others to the channel and i can add them to the script @ tag later [01:48:52] i still cant find the setting to mute channel notifications but allow @ tags to notify [01:49:00] which would be handy and is why i did it like this really [01:49:10] ivemuted the channel and still received the notification [01:49:18] okay cool [01:49:19] nice [01:49:25] https://cdn.discordapp.com/attachments/416000318149754881/659920738639151115/unknown.png [01:49:33] perfect, because you don't want to care about all teh successful alerts every 10 minutes [01:50:01] i thought i wasnt getting the alerts when i had it muted, but idk [01:50:28] if you have some time could you have a look at pr 656? [01:50:44] yeah [01:51:42] im still pretty AWOL with all this family stuff for the next few days at least, but i have a little bit of time this afternoon [01:52:04] so ill try to have deep read of it [01:52:20] what is the plan, do we need to put out 4.7.3 patch with the fix? [01:52:36] ah its a pretty small PR [01:52:39] yup [01:52:44] should definitely get through this [01:53:01] ill do the code review, and i read that salmon is doing some user testing [01:53:05] so hopefully that covers us [01:53:34] i can test to compile and run the node and run any code tests but i dont think i have time today to do a full battery of user testing [01:53:41] Testing syncing [01:53:50] syncing is basically what needs testing [01:53:54] okay [01:53:57] i can do that [01:54:03] just wipe datadir and do IBD [01:54:05] ? [01:54:21] i guess youll have a clearer picture after reading the text, let me know if u have any questions. would be great if this is done before i leave on holidays [01:54:29] yeah for sure [01:54:55] the most critical scenario is nodes in the range 2 days-a few hours from tip [01:58:42] this can be easily tested using the bootstrap which is 2 days old [01:58:48] thats how i tested it [01:59:12] got it. already tested syncing a week from tip and it's good. syncing from scratch looks good so far too. will now get in the 2 day testing [02:02:13] im building now [02:02:40] will post here later and comment any questions on the github PR [05:05:33] made a couple minor comments/questions on the PR running some syncing tests now [05:07:45] just to confirm, you mentioned up there ^ somewhere that it doesnt account for port right? so changing port on the same IP won't grant another 4k blocks of header spam correct? [05:22:58] Peers are discerned by their ip address, this means peers sharing ip address will also share the same points list. This can be changed with -headerspamfilterignoreport (default: true). [07:18:08] My node seems to have stopped syncing last time I checked. Some issue with an invalid previous block, not the spam filter though as far as I could tell from a quick skim of the logs [07:18:37] Will look a bit more deeply soon. Might try resync from the beginning as there was already some block data in that folder [07:24:34] ive backed up that data dir in case its of interest and started a natural sync from block 0 [07:29:21] my syncing from scratch node gave me this error message and then closed [07:29:34] terminate called after throwing an instance of 'std::ios_base::failure[abi:cxx11]' [07:30:03] 🤷‍♀️ [07:30:09] weird error [07:30:55] debug log looks normal except for the last line [07:30:57] 2019-12-27 05:05:41 UpdateTip: new best=b95e7593fe09f65ac78a2f57dc2303a809061144fa926a45f9a18576e1dbd17a height=3768909 version=0x7135e1ff log2_work=73.214486 tx=7842332 date='2019-12-27 03:50:08' progress=0.999903 cache=163.5MiB(727363tx) 2019-12-27 05:05:41 UpdateTip: new best=75e1b33efe3b83ccf71c67101ddfeddbd6bfd0507d71d3f8180c30a7b0246a36 height=3768910 version=0x7135e1ff log2_work=73.214487 tx=7842334 date='2019-12-27 [07:30:58] 03:50:24' progress=0.999903 cache=163.5MiB(727365tx) 2019-12-27 05:05:41 UpdateTip: new best=0e69915665886606083408e023a424cddb3beef0a00fff2a459c8dc043041259 height=3768911 version=0x7135e1ff log2_work=73.214487 tx=7842336 date='2019-12-27 03:51:12' progress=0.999904 cache=163.5MiB(727367tx) 2019-12-27 05:05:41 AdvertiseLocal: advertising address 173.174.6.91:44440 [07:31:10] https://en.cppreference.com/w/cpp/io/ios_base/failure [07:31:59] what OS/version are you using? did you compile it yourself or use the nav.community binaries? [07:32:17] i compiled it myself and it's ubuntu 18.04 [07:32:28] it's a virtual mahcine [07:32:59] but my windows is working normally [07:33:29] are you using the ubuntu system dependencies? or building the depends folder as well? [07:33:45] i built the depends folder [07:33:56] and then configured to point at the depends folder right? [07:34:02] yeah [07:34:06] okay [07:34:30] do you know what version of gcc compiler you're running? [07:34:47] gcc (Ubuntu 8.3.0-6ubuntu1~18.04.1) 8.3.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [07:35:30] did you compile the depends just before compilign navcoin or was it compile some time ago? [07:35:42] it was compiled some time ago [07:35:52] i just kept using the same depends [07:36:08] some stackoverflow issues are related to libraries being compiled with an older version of GCC than is currently installe [07:36:19] i dont know why these changes would effect that though [07:36:37] hmmm [07:36:52] have you tried again? [07:36:59] im hesitating [07:37:08] wondering if i should just leave it [07:37:09] maybe try run make in the depends folder, recompile navcoind and try again [07:37:23] or wait for an adult to answer your questions [07:37:25] (alex) [07:37:31] lol [07:37:31] 😅 [07:37:36] the node was syncing fine all day long [07:38:12] was it fully synced and just processing new blocks when it happened? or still in the process of catching up to the latest block? [07:38:36] it was in the process of catching up to the latest block [07:38:50] and from the looks of it, it was only a couple of hours behind [07:38:57] okay [07:39:02] maybe wait for alex and see what he says [07:39:13] because that is the critical time where these changes are in effect really [07:39:13] sure [07:39:38] yeah [07:40:07] im not going to be online tomorrow as that is when my step-dad's funeral is [07:40:17] and i will be with my family for the entire day [07:40:42] but im on my couch for the next few hours so maybe we will catch alex before i go to bed [07:40:52] roger that. i hope you and your family is doing okay. [07:41:12] thanks, yeah its pretty tough time tbh [07:41:16] we're surviving [07:41:28] no doubt, can't even imagine. [07:41:37] too young! he was only 66 [07:41:45] his mum is still alive even, she's 93 [07:41:52] wow. [07:41:58] really sorry to hear man [07:42:01] so a definite shock to everyone [07:42:10] yeah, anyway im derailing this channel [07:42:21] my sync is going again ill report back before i go to bed [07:42:22] 🙂 [07:42:33] okie dokie [10:10:53] syncing still going [10:10:57] only at like 15% [10:10:59] im off to bed [10:30:53] maybe its because of the std::inserter [11:03:48] @prole i think the telegram bot crashed [11:29:48] i've commited a possible patch @salmonskinroll [12:05:44] @aguycalled maybe we should add a header spam test to the cpp test suite? [12:07:04] Or would it be possible to create a test scenario in the py suite? [12:35:37] yes sure [12:36:10] on the py suite it can be done with a mininode [12:37:00] https://github.com/navcoin/navcoin-core/blob/master/qa/rpc-tests/test_framework/mininode.py [12:42:38] I see, I'll poke around and see if I can make a test that makes sense [17:40:07] morning guys. building the latest commit and will do a full sync [20:19:45] my tests were successful with the last patch, ive been working today trying to get dao_extension back on track [21:45:33] Segmentation fault (core dumped) ./src/qt/navcoin-qt [21:45:44] latest commit on the headspam branch [21:45:53] im running it with debug=1 now to see what's up [21:45:59] debug=1 wont say anything [21:46:11] it needs to go through the qt debugger or gdb [21:46:20] and ./configure --enable-debug [21:46:27] ok [22:40:40] running two nodes with qt debugger now, hopefully it'll find something soon