Patchwork: My ssb broke... Error: when trying to retrive:log

[marylychee]

Accidentally opened both patchwork and patch bay at the same time, so I think in both clients trying to index, things crashed kaboom-ed.

Error: when trying to retrive:log,1560666067414at since:680417528
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/flumeview-level/index.js:139:28
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/flumedb/index.js:70:17
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/flumelog-offset/inject.js:75:17
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:48:11
    at Object.read (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:49:14)
  Error: past end:680414149-2431614458 < 680418068
    at Object.read (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:49:17)
    at next (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:47:16)
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:43:11
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:96:11
    at get (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:29:7)
    at Object.readUInt32BE (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:93:9)
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:41:16
    at /Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:96:11
    at get (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:29:7)
    at Object.readUInt32BE (/Applications/patchbay.app/Contents/Resources/app.asar/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:93:9)

I think I've done before.... and now again stuck on forever indexing on patchwork, and get this error in patchbay.

What's the best course of action from here? Should I just delete some of the folders from .ssb except my keys and stuff, and re-download everything from the network like I did last time???

[christianbundy]

Oh no! Could you post the error message you get from Patchwork when trying to open the latest release? There are a few things that could be going wrong and I don't want to say "delete your flume stuff!" unless that's actually the best course of action.

If you are interested in deleting your views, there are commands here. You won't have to download new data from the network (:tada:), this just deletes your database indexes so they'll regenerate.

[marylychee]

no worries, @pospi in the end helped me out, so we deleted everything except from gossip, config and secret - sorry I didn't see your helpful suggestion first!!

luckily i was sitting next to Teq then in a local wifi, so I think I redownloaded everything mostly from them, relatively faster than last time this happened.

Super helpful to know there is a way for next time this might happen!

[marylychee]

ok things went whack again...

I went ahead and tried your suggestions with:

$ find ~/.ssb/flume/ -mindepth 1 -maxdepth 1 -type d -ok rm -rf {} \;
< rm ... /home/christianbundy/.ssb/flume/search > ? y
< rm ... /home/christianbundy/.ssb/flume/feed > ? y
< rm ... /home/christianbundy/.ssb/flume/query > ? y
< rm ... /home/christianbundy/.ssb/flume/private--oaWWDs8g > ? y
< rm ... /home/christianbundy/.ssb/flume/backlinks--oaWWDs8g > ? y
< rm ... /home/christianbundy/.ssb/flume/time > ? y
< rm ... /home/christianbundy/.ssb/flume/links > ? y
< rm ... /home/christianbundy/.ssb/flume/clock > ? y

and deleting ~/.ssb/flume/log.offset

I think I get this problem everytime I use patchbay from the repo folder & maybe i just go npm run dev
and I think I forget to run the sbot first?

Running the desktop app

Easy mode (embedded sbot):

# from the patchbay repo folder
npm start

Harder mode:

sbot server
#from the patchbay repo folder
npm run dev

anyways I think something is happening? I don't know

[marylychee]

I also ran npm run dist following @mixmix suggestion here

the first one might be fatal.

[christianbundy]

Hmm, you shouldn't have to delete your offset file, was it throwing errors in Patchwork that made you have to delete that? The only useful data point is that I think Patchwork is using libraries that are more up-to-date than Patchbay, but that's all I've got.

Happy to hop on a call with you and debug if you'd like, it's a bummer things keep breaking. :/

On Sat, Jun 22, 2019, at 17:31, Mary wrote:

I also ran npm run dist following @mixmix https://github.com/mixmix suggestion here https://github.com/ssbc/patchbay/pull/348#issuecomment-504633867

the first one might be fatal.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/ssbc/patchwork/issues/1107?email_source=notifications&email_token=AAEDIZDLT3E6RTSMLEOKYATP3275BA5CNFSM4HYQSDZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYKT3IY#issuecomment-504708515, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEDIZHWEPWBMJV6J5PTAATP3275BANCNFSM4HYQSDZQ.

[marylychee]

Ok, upon opening and closing Patchwork a few times I think it's indexing!!

Thanks helping out! Yeah it was a bummer. For next time, how do I see errors in Patchwork? I deleted it because doing the other flume things didn't seem to work.

It's interesting as I definitely have had this problem at least 3-4 times in the last months and most of the time, it was because I'm jumping between Patchbay & Patchwork.

[christianbundy]

I think you can right-click anywhere on the screen and hit "Inspect element", then hit the "console" tab, and the errors should be there. Alternatively you can run from source and npm start and errors should be output to the console.

On Sat, Jun 22, 2019, at 19:00, Mary wrote:

Ok, upon opening and closing Patchwork a few times I think it's indexing!!

image https://user-images.githubusercontent.com/18238352/59970590-b6c8bb00-95ad-11e9-8305-9a7a089014f8.png

Thanks helping out! Yeah it was a bummer. For next time, how do I see errors in Patchwork? I deleted it because doing the other flume things didn't seem to work.

It's interesting as I definitely have had this problem at least 3-4 times in the last months and most of the time, it was because I'm jumping between Patchbay & Patchwork.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/ssbc/patchwork/issues/1107?email_source=notifications&email_token=AAEDIZD6HZGAI27XAJ7CCZLP33KMTA5CNFSM4HYQSDZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYKUZYQ#issuecomment-504712418, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEDIZGR3YZEVAFUQBUQ4ADP33KMTANCNFSM4HYQSDZQ.

[marylychee]

ohkay. thank you!! I think I now just have missing messages... in multiple threads. :O did I broke broke it, when it already broked?

[marylychee]

ok really frustrating :'( I'm trying to write a post, and reference missing posts, ... and organise ssb website development, and kind of feel like I'm roadblocking everyone as my SSB is broke.

[marylychee]

could I possibly have forked my feed?

[christianbundy]

The last post I see from you is %dUC67XtTvRkTTF971zjcMzCMlIKzz8QBSUJKe45lgtM=.sha256, is that correct?

[marylychee]

oh no.... that one is 4 days ago.

what I see is last post is a private message w/ endo 17 hrs ago
and a public post 18 hrs ago...
(Using patchbay to more easily see what I last posted)

[marylychee]

and 1 day ago...

%QO86kITfSin3xqlNswARuoVjaVoEmsL7dvaxqNyzrlk=.sha256

3 likes on it??

[mixmix]

I see a message from you 2 hours ago : %Yupg2opyOct6JcDrRWyf60wm0gXeW1A6UzJO9/Z49Gk=.sha256

So I don't think you've forked your feed.

Really do not delete your log.offset file again without checking with a scuttlebutt dev. It's very very rare you should need to do that.

[mixmix]

it's still possible you are forked for some people, but very unlikely.
The way you would be forked is if you started posting before you had aquired all of your messages back again. That's unlikely because at the moment patchwork is pretty much locked up completely till indexing is all done ... patchbay you might have been able to find somewhere not locked up to post.

For reference if you start Patchwork and Patchbay both at the same time with npm run start or with installers, then they bounce off each other, and may partially run, but they won't mess anything up irrevocably. It's safe. With that npm run lite or npm run dev ... I haven't looked to see if patchwork hosts all of the plugins needed in patchbay at the moment ... actually I know it doesn't (ssb-meme isn't in patchwork), so I would expect that if you try to use that some pages will crash (and might crash the server).

I highly recommend only running one at a time. If you want the same ocnfig (like hops 2) you can set that.

[christianbundy]

I see you too -- sorry, I think Patchwork was hiding messages? I'm not sure, I still haven't wrapped my head around the feed summary stuff.

Those "cannot load thread" things are because you deleted the thread from your log.offset, sometimes it takes a while to load them back but they should come back. I don't think you've forked your feed either, you're good!

[marylychee]

damn. yeah ok I wont touch log.offset again.

the cannot load thread is confusing. @mixmix what do you mean as locked up? even private messages? Because maybe that's where I was impatient and still sent things?

like as to why im not sure, I can load Mix's profile post and it stops here %t4W5WtMY4Wknw+lIdz5WBkOWSooQL7j7znWbjqA+aE8=.sha256 - so max posts im getting up to is 6 days ago

where as with Mikey's profile - I have infinite load like ajax effect reaching (at least?) up to 6 months ago.

Is there anything I can do to fix? I'm very sure I ran npm run dev

[marylychee]

like so much of this is happening

[christianbundy]

This is probably a silly question but have you restarted Patchwork? Maybe check ~/.ssb/gossip.json and ensure that it's not empty? Not sure if you deleted that but if so maybe your Patchwork doesn't know which peers to connect to?

[pospi]

This could be unrelated bugs. I've been getting a lot of broken things in
Patchbay since v8 was released- often don't see any friends or follows,
have empty inbox, that kinda stuff. When it happens I usually just kill and
reopen the app and sometimes that helps, sometimes not. The same "have you
tried turning it off and on again" approach does not seem to be working
here :/

On Thu., 27 Jun. 2019, 11:37 am Christian Bundy, [e-mail]
wrote:

This is probably a silly question but have you restarted Patchwork? Maybe
check ~/.ssb/gossip.json and ensure that it's not empty? Not sure if you
deleted that but if so maybe your Patchwork doesn't know which peers to
connect to?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ssbc/patchwork/issues/1107?email_source=notifications&email_token=AADK2JB4LIYMSY67RX52XFTP4QKVRA5CNFSM4HYQSDZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYVIS6I#issuecomment-506104185,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADK2JAVWNAQOCZBNEWXPEDP4QKVRANCNFSM4HYQSDZQ
.

[pospi]

@mixmix maybe a useful debugging step here would be isolating such problems
to the UI or DB. If some of these "missing message" posts can be found in
the DB then that makes the cause of the bug clearer. It's also unclear
which parts of the DB are specific to Patchbay, and which to Patchwork.
Certainly it seems like different clients give different results.

The contents of gossip.json also sounds quite useful, maybe there's a
lack of peers near the root of the problem. These bugs have been happening
persistently, and @marylychee has restarted her clients and wiped
everything bar secret & gossip.json and done a full resync more than
once. None of that seems to have solved the problem, and she's not trying
to do anything fancy like running sbot separately- just running the apps as
packaged.

On Thu., 27 Jun. 2019, 12:26 pm pos pi, [e-mail] wrote:

This could be unrelated bugs. I've been getting a lot of broken things
in Patchbay since v8 was released- often don't see any friends or follows,
have empty inbox, that kinda stuff. When it happens I usually just kill and
reopen the app and sometimes that helps, sometimes not. The same "have you
tried turning it off and on again" approach does not seem to be working
here :/

On Thu., 27 Jun. 2019, 11:37 am Christian Bundy, [e-mail]
wrote:

This is probably a silly question but have you restarted Patchwork? Maybe
check ~/.ssb/gossip.json and ensure that it's not empty? Not sure if you
deleted that but if so maybe your Patchwork doesn't know which peers to
connect to?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ssbc/patchwork/issues/1107?email_source=notifications&email_token=AADK2JB4LIYMSY67RX52XFTP4QKVRA5CNFSM4HYQSDZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYVIS6I#issuecomment-506104185,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADK2JAVWNAQOCZBNEWXPEDP4QKVRANCNFSM4HYQSDZQ
.

[marylychee]

@christianbundy yes restarted by closing and reopening a bunch of times and ~/.ssb/gossip.json not empty.

The error on patchbay

I dont know... I think it's not UI @pospi ??

at this rate should I try deleting everything but secret & gossip.json again, as this time the unique problem was weirdness was from rm -rf alot of things from ~/.ssb/flume/

The infinite indexing error starts when I switch between patchwork/patchbay/commandline patchbay run dev too much.

[christianbundy]

Hey have you been running Patchwork from master or the latest released version?

[marylychee]

latest release. Version 3.14.1

[christianbundy]

Sweet, thanks. And have you deleted and rebuilt your indexes since you last opened Patchwork and Patchbay at the same time? Just trying to get a feel for what the current situation is, I'm really not sure why this would be happening.

[pospi]

I don't think you should be doing npm run dev either. Not sure who gave
that advice but it seems like a bad idea as it means running sbot
separately, doesn't it? I would only ever use npm start... keep it
simple...

On Thu., 27 Jun. 2019, 3:21 pm Christian Bundy, [e-mail]
wrote:

Sweet, thanks. And have you deleted and rebuilt your indexes since you
last opened Patchwork and Patchbay at the same time? Just trying to get a
feel for what the current situation is, I'm really not sure why this would
be happening.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ssbc/patchwork/issues/1107?email_source=notifications&email_token=AADK2JH4NFD6F4AAWK3QOPLP4RE4FA5CNFSM4HYQSDZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYV6SDA#issuecomment-506194188,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADK2JDOZEEG7M4GADJJA4TP4RE4FANCNFSM4HYQSDZQ
.

[marylychee]

yes. I've just tried deleting and rebuilding my indexes. It was weird that it seemed to have finished rebuilding them really fast... and then so I've checked and still a bunch of missing messages. 🤷‍♀

[marylychee]

yes i didn't know that npm run dev needed sbot. i was developing so I thought that was better option. sighs

okie dokie - I'm doing the thing. I'm doing to the whole delete everything but secret and gossip.json again. (and by delete I mean mv into another folder just in case)

[marylychee]

OK FIXED <3 THANK GOODNESS.

have missed u all