App: Missing tx history

[SpecialAgentX]

_defi-app: 2.1.17_

As I wrote in bug #515, the backup export in version 2.1.17 (MacOS client) does not contain transaction history (I restored it on an Windows 10 PC). Furthermore: Restore (on Windows PC) with a seed does not contain tx history as well.

So, in my eyes there is NO possibility to backup or restore TX history. This is critical, because if one lost his original defi-app installation, there is NO WAY to provide exact tx data for financial institures e.g.

So, this needs to be fixed as high priority bug (in my eyes).

Absolutely a risk for the whole defichain project.

[izzycsy]

Hi @SpecialAgentX , thanks for the feedback, please give us time to investigate this issue. Have you tried to restore wallet using your wallet.dat file?

Windows — C:

\Users\\AppData\Roaming\DeFi Blockchain\walletswallet.dat

Mac — Go to Folder:

/Users//Library/Application Support/DeFi/wallets/wallet.dat





[SpecialAgentX]

Hi izzycsy,

so, you mean just copy the wallet.dat form mac to windows without the built in backup functionality? I'll try this in a couple of days.

Thx.

[izzycsy]

Hi @SpecialAgentX , you didn't state which kind of operating system you're using, hence I'm providing the respective routes for you to find your wallet.dat files. However, you should also be able to restore wallet using wallet.dat if in the event you're using 2 PCs.

Making a copy of the wallet.dat is for your own keepsake. In the latest version v2.3.0, when you Restore Wallet, it'd be via seed / wallet.dat.

[salomonkarsten]

@izzycsy In Version 2.3.0 the Transactions are not visible. Therefore all data missing for the .csv export. .csv only shows a header. I wonder why it is not possiblt what another project like https://github.com/DeFi-PortfolioManagement/defi-portfolio/blob/master/README.md

can do. Would it not great to fuse both projects?

[izzycsy]

Hi @salomonkarsten , may I have your app logs and a screenshot of the issue you're facing please?

Thanks for sharing the project, we appreciate users creating their own community project. The original creators could perhaps share it on DeFiChain Telegram (EN) to stir some interest in the mean time 😉

[salomonkarsten]

@izzycsy

AppBinaryLogs.log
Applog.txt.log

[salomonkarsten]

@izzycsy no history is displayed correctly none of the latest transactions are displayed.
But i can see them in the history commands.

[izzycsy]

Hi @salomonkarsten , can you send your logs please? The team is investigating the issue, thank you.

[salomonkarsten]

@izzycsy habe a look, they are send already

[izzycsy]

Hi @salomonkarsten @SpecialAgentX , does the issue persist in v2.3.1? When you've updated and if the issue persists, please send screenshot again + the latest app log, bin log & app config, thank you.

[salomonkarsten]

AppBinaryLogs.log
Applog.txt.log
@izzycsy problem remains, i see only transactions older than 13.1.2021 also i do some daily.

[SpecialAgentX]

Hey,

yes problem persists in 2.3.1:

I have backuped my wallet on my Mac OS client (v2.3.1) (this is the original with tx history). Installed 2.3.1 on an Windows 10 PC, reindexed everything and then imported the wallet.dat from MacOS by first copying it local to Windows PC and then used the UI restore button.

Im seeing on Windows PC no transactions for every wallet except of my DFI wallet. All or some (did not check for completeness) tx are there for DFI wallet. Nothing for LTC, BTC, USDT. The amount of all wallets are correct displayed.

I attached app logs from Windows PC.

Regards

applogs.zip

[SpecialAgentX]

Hi @SpecialAgentX , you didn't state which kind of operating system you're using,

I've updated my bug description. Sorry.

[izzycsy]

Hi @salomonkarsten @SpecialAgentX , okay so you've manage to restore wallet, just that you have missing tx history. Can you try Reindex Option via Settings? After that, if issue persists, please send your app log, binary log & app config again.

[SpecialAgentX]

@izzycsy : This did it! I do have some tx data within other wallets as DFI. Thank you!

But it seems, that some tx are missing on Windows:

In my ETH wallet, when Im scrolling through tx pages, Im seeing that there must be 48 tx but on page 3 defi-app tells me that there are no tx. See screenshot 1 and 2

EDIT: same on Mac OS wallet. Seems to be a general problem and not related to backup/restore.

Pressing the arrow button shows this:

[izzycsy]

Hi @SpecialAgentX , the team has halted v2.3.1 at the moment due to some issues, your issue is not related to it, but I suggest you wait for v2.3.2 and try Reindex again. I'll update you again when the team fixes it and is ready for release.

[luckythai]

It's a problem similar to mine.

My problem is still there. Only the last 3 rewards are displayed in the app under transactions. They are available when the data is exported. both in the app 2.1.17 / 2.3.0 / 2.3.1 / Wallet - 2.3.2

https://github.com/DeFiCh/app/issues/257

[izzycsy]

Hi @luckythai , v2.3.2 is not officially release yet, please refrain from downloading it until official announcement on DeFiChain Announcements (EN). If you've already installed, please wait till we officially release it and see if your issue still persists. Thank you.

[izzycsy]

Hi @SpecialAgentX @luckythai , update v2.3.2 is out, this is a mandatory update. 
Please read through the release notes before updating.

Do give a thumbs up to this comment if your issue has been resolved, otherwise, please send new information (include screenshots) and app logs. Thank you.

[salomonkarsten]

Update and reindex didn't solve the problem. I wonder why the portfoliomanager app can Retriever the data but the app cannot

[izzycsy]

Hi @salomonkarsten , thanks for updating, may I know which tx history are you missing? Can you send your app logs please? Thank you.

[luckythai]

Hi izzycsy,
with dfi only the last 3 rewards in the history are shown. On the inventory and when exporting the files everything is ok.
Screenshots are in issues # 257, are the same, only the date changes. Everything is ok, only the dfi reward is not shown except for the last 3. https://github.com/DeFiCh/app/issues/257

Logs.zip

[izzycsy]

Hi @luckythai , thanks for your feedback. When you export the CSV files your tx history is displayed, only in the app UI it is not? Only for DFI? Do you have a screenshot? Logs well received.

Hi @salomonkarseten , can you try refresh utxo via settings?

[luckythai]

Hi izzycsy,

yes exactly, only with dfi and only the reward is a problem, but only the display of the transactions. I only ever see the latest 3.
with refresch the latest rewards again (if one is paid out in the meantime). The credit is displayed correctly.
It's not a problem for me now, because the export works.

refresh utxo does not bring any improvement

[luckythai]

Here is the screenshot after refresh utxo

Everything is displayed correctly, every transaction, but the rewards are only ever limited to the last 3 that I have received.
But I should normally see "hundreds".

[izzycsy]

Hi @luckythai , thanks for the screenshots. Please give us time to investigate this. Thank you.

[dnamurphy]

@izzycsy - this issue is also found in 2.3.3 (MacOS).

I have a Windows 10 machine running 2.3.2 and MacOS 2.3.3. The latter was freshly (re)-installed (I wiped 2.3.2), and synced yesterday, accelerated with the snapshot. Swap and LM records are missing as before under Mac (Windows is OK).

I also note that when I add liquidity in Windows, the transaction does appear in the Mac client once the block height increases.

Hope this helps.

[izzycsy]

Hi @dnamurphy , thanks for your feedback, please give us time to investigate this issue.

[luckythai]

@izzycsy
now with the 2.3.3 the rewards work again in the view, but now all the transactions in the wallet are gone.
have not yet assigned a password and every backup of my wallet shows different transaction sizes when importing. But you only see a maximum of 30 transactions. With some wallet backups even only 10. The rewards are now displayed normally again.

[izzycsy]

Hi @luckythai , I'm not sure how you manage your wallet.dat backups, but personally I label them before I save the backup. I understand that it's a bit late to do that now, but in the future maybe you can practice this so you don't lose track of your own file versions. Maybe you can try using the latest wallet.dat and just continue using it from there. During every upgrade, the app is trying to find past transactions amongst other transactions in the each block. Are your addresses and balances are reflected correctly?

[luckythai]

Hi @izzycsy
yes my addresses and balances is displayed correctly everywhere.
Except for restore with the 24 word seed. There is no address there, only the balances is right here.
It is not a problem of the organization of my wallet (see picture) and also not a user error. By fixing the previous problem (only 3 rewards are displayed) a new error has arisen.

With every backup that I import, the correct balances and addresses appear. The reward appears correctly with every import wallet. The only difference is the amount of transactions in the hystery. Only the very first transaction is shown everywhere. Any subsequent transactions will no longer appear.
The transactions do not appear in the CSV export either.

For a wallet, the amount of transactions is 3
With the other 60, with the other 220 .... all different values.
The database is the same with every import.
But the error is also when using 2 or 3 computers and different wallets.

An import with the old txt backup always breaks off with the error of a missing dump file. Only the import with the dat file and the 24 words seed was possible for me. My wallet doesn't have a password yet.

[luckythai]

There are 2 different errors in the display in the wallet 2.3.3 and the export of the cvs. And 1 error with the restore of the txt file.
I think the display errors comes from the app in the created " ~/.config/defi-wallet " which is lost when changing the pc or reinstalling the system, or by deleting it.

1. Restore temporally different backup dat files from the same wallet.

Error -> Different amount of transactions with and without reward. The CSV export adapts to the displayed amount and not to the actual transactions.
Backups from Wallet 2.3.0 show the same amount of transactions, but a new restore with the 24 words seed has a different amount of transactions.
The amount of transactions of the rewards are different everywhere.

2. A restore of a wallet dat and a restore of 24 words seed. Every restore.

Errors -> e.g. 210 transactions displayed, but only the first 10 can be seen. when scrolling to the 2nd page there are no transactions. The transaction overview with reward works without any errors. But not all that were actually received with the wallet. The new ones are all there.

3. a restore of the txt file is not possible with the interface.

He breaks with the error> missing dup file

[luckythai]

1. Restore different backup dat files from the same wallet. Error ->

Bevore 2.3.0

From wallet 2.3.0 and more

From 24 words seed

[luckythai]

2. A restore of a wallet dat and a restore of 24 words seed. every restore.

Errors ->

[izzycsy]

Hi @luckythai , thanks for the elaboration. Let's do away the Restore via mnemonic seed & any.txt file. And narrow down on what matters, which are the wallet.dat and your passphrase from v2.3.0. The team has disable mnemonic seed generation for security purposes. We'll look into the missing tx history issue for wallet.dat. Thank you for your understanding.

[luckythai]

Hi @izzycsy
the Restore via mnemonic seed & any.txt file, was just an attempt to further isolate the error.
so the dat file is not the cause, but the app. if the dat file were the cause, the import with the 24word mnemonic seed should have had the correct transaction display, but it made the same display errors of the transactions, although the dat file created by the app.
Somewhere the database is no longer queried, but only the content of the dat file and/or the ./config/defi-wallet folder, which is just not complete. Otherwise the export would be complete. So the dat file has too little information or it is only queried from the cache and not from the database. When the wallet is restored, not all information from the database is restored, but rather the current status of the computer and fragments of database information.
However, no inventory or value is lost from the wallet; it only affects the display of transactions and the csv export.
But I am firmly convinced you and the others of the team will find a solution with the information.

[salomonkarsten]

On top even New transactions are not listed in the tx history. It seems not all ismine addresses are taken into account for tax history. Did you ever look at the dfip. You should include this Grat apps as these can evaluate in a correct manner the history. I wonder why the wallet cannot do the same and just use the history command!!!

[luckythai]

Hi @izzycsy
Ok, proceed differently and let's focus on the dat file.

Starting position

1 computer with ubuntu 18.04 and cli 1.6.3 - full node.
Database of node 1.6.3. Was created without a snapshot and after Defichain.
2 computers with windows 10 and 2x Ubuntu 18.04 as WSL for the Defichain app 2.3.3 Appimage.
Fullnode's cloned database. 41.6 GB (I call the database node.1.6.3)
1 computer (I call it original now) with windows 10 and ubuntu 18.04 WSL Defichain App 2.3.3 Appimage. Database created with reindex and was 80 GB in size. (I call the database node.1.6.2)
#
Only 1 dat file was used. No mixing of backups and with the same cloned "~ / .config / defi-Wallet" folder. from the original PC with the 80 GB database - node 1.6.2 reindex. The defi.conf is the same and also the first 3 entries on every computer.
This datfile showed 5 transactions. No receiving addresses.

2 computers with the cloned database - node 1.6.3 only showed 3 transactions. No receiving addresses.

Why? is completely cloned.
The difference is the database. OK. Clone the 80GB node.1.6.2 database and see there 1 computer now also shows 5 transactions and the 2nd with the database node.1.6.3 only 3 transactions.

On the computer with the three transactions (node ​​1.2.3) I manually entered every address with credit that I found with "listadressgroupings". 17 addresses which I have not generated before.
Transactions were added with each address, until after the last address it increased from 3 transactions to 210.

So now I have 2 databases node.1.6.2 and node 1.6.3 and 2 Wallet.dat. an original one (I call it wallet.dat-5adress) and one where the receiving addresses were entered (I call them wallet.dat-3adress)

CVS export only the displayed amount of transactions. So it is not exported in the real amount.

I send the result with pictures and log files of every procedure and computer in zip format.

Node.1.6.2

Node.1.6.3

no receiving addresses

the receiving addresses were entered

[luckythai]

Original PC. Node 1.6.2 reindex - all clone from this PC.
Everything was created on this PC and the 2nd wallet and the 2nd database were not used. The evaluation of this pc was made at the end. 3 new transactions have been added that I have made in the meantime.
The addresses entered do not appear there. The new transactions and reward are displayed.

Node.1.6.2.original.zip

[luckythai]

PC no1. Evaluation of Node 1.6.2 and Node 1.6.3 and the 2 wallet.dat 3 + 5 addresses.

Node.1.6.3.Adress3.zip
Node.1.6.3.Adress5.zip
Node.1.6.2.Adress3.zip
Node.1.6.2.Adress5.zip

[luckythai]

PC no2. Node 1.6.3 and the wallet.dat 3 addresses. chance Hardware

Node.1.6.3-chance.HW.zip

[izzycsy]

Hi @luckythai , thank for the updates and feedbacks, please give us time to investigate.

[memzila]

Are there any news about the issue? I have the same that i don't see any new rewards or transaction since i have tried to move all to a new address. Since this date, i have no newer transactions shown in the wallet app. I see only transactions until the date when i have send all to new address. In the previous mentioned portfolio app, all transactions are shown correct. I have also tried on another pc with the snapshot download. With that i see newer transactions since the snapshot creation in the wallet, but nothing older then when the snapshot was created. So the whole transaction view in the app need a good update. I suggest to get some help from the portfolio developers

[izzycsy]

HI @memzila , the team is still investigating the missing tx history issue. May I know when you export the data, are the tx history reflected?

Could you save your app logs, .zip it and send them please? Thank you.

[memzila]

HI @memzila , the team is still investigating the missing tx history issue. May I know when you export the data, are the tx history reflected?

Could you save your app logs, .zip it and send them please? Thank you.

Hi @izzycsy Bella. In Export File looks like all transactions are visible.
About logs. Do I need to remove something for them before uploading here?

[gamsterx]

You can add me to the list of affected users.

It happened after the following interaction:
Remove liquidity from USDT-DFI pool (100%)

Notes:

• Since removing the liquidity from one pools no rewards are shown in the tx list.
• Other types of tx are also missing (swap, addToPool, ...)
• I have the app running on two machines (win + mac) both apps show same behavior
• I can see all tx with the "defi-portfolio" app

[izzycsy]

Hi @memzila , no need.

Hi @gamsterx , thanks for your feedback, yes in the mean time you may use DeFiChain Portfolio to view your tx.

[memzila]

Here the logs
Logs.zip

[izzycsy]

Hi @SpecialAgentX @salomonkarsten @luckythai @dnamurphy @memzila @gamsterx , v2.4.0 is out, this is a mandatory update. Please read through the release notes before updating. _This comment is to notify you about the update._

[memzila]

Hi @izzycsy Bella. Looks like in the new version no longer Transaktion History is in the chain. Is it planned to address that in future updates of the app?

[izzycsy]

Hi @memzila @nippur72 , yes the team is investigating. In the mean time you can use , in the mean time you may use DeFiChain Portfolio to monitor your tx history.

[gamsterx]

Upgraded to 2.4.0.

• I synced with a snapshot --> even more txns and rewards missing
• I did a full reindex (took about 30h)--> same effect. Midway through the reindex I was seeing all the txns and rewards, but when the reindex was complete they were gone again.

[memzila]

Hi @memzila @nippur72 , yes the team is investigating. In the mean time you can use , in the mean time you may use DeFiChain Portfolio to monitor your tx history.

Hi @izzycsy bella. I don‘t think the portfolio app is still working, when all the rewards transactions are no longer in the blockchain.

[izzycsy]

Hi @SpecialAgentX @salomonkarsten @luckythai @dnamurphy @gamsterx @memzila , thanks for the update, v2.4.2 is out, this is a mandatory update. 
Please read through the release notes before updating. _The above issue is not included in this update yet, this comment is to notify you about the update._

Our team is investigating on the missing tx history issue, will keep you updated here or you may refer to DeFiChain Announcements (EN) for a quicker notification update. Thank you for your patience.

[chrsch]

I'm also missing TX in the wallet. If you need input from another wallet installation just let me know @izzycsy

[izzycsy]

Hi @SpecialAgentX @salomonkarsten @luckythai @dnamurphy @gamsterx @memzila @chrsch , v2.4.3 is out. Please read through the release notes before updating.

Do give a thumbs up to this comment if your issue has been resolved, otherwise, please send new information (include screenshots) and app logs to this issue v2.4.0: Missing past tx history #891. We can further discuss on the issue there, as we would like to keep same issues in one thread. Thank you.

[nippur72]

@izzycsy I upgraded to 2.4.3 but the history is still missing.

Here are my logs: logs.zip

[chrsch]

For me with 2.4.5 TX are still missing.

Hint: Seems to apply only for wallets that are imported. @izzycsy