Skip to content

QML Load Snapshot Signet (160,000 height) #424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

D33r-Gee
Copy link
Contributor

@D33r-Gee D33r-Gee commented Oct 11, 2024

Open for review and testing only. Do not merge!

GUI Integration for UTXO Snapshot Loading

Overview

This PR adds initial GUI support for loading a signet UTXO snapshot at height 160,000, building on Bitcoin Core's assumeutxo infrastructure.

What This PR Does

  1. Adds a basic GUI interface for loading UTXO snapshots via the Connection Settings panel
  2. Implements non-blocking snapshot loading via QT Thread
  3. Provides visual confirmation of snapshot activation
  4. Adds a snapshot progress notifications accessible via a handler

Implementation Details

Core Components Modified

  1. Node Interface (src/node/interfaces.cpp)
  • Implements snapshotLoad() based on the loadtxoutset RPC
  • Uses background QT thread for non-blocking operation
  1. QML Integration (src/qml/models/nodemodel.cpp)
  • Manages snapshot loading state and progress
  • Introduces a worker class to manage snapshot metadata
  • Uses Qt's thread system to prevent UI blocking
  • Provides progress updates via Qt signals
  1. UI Components (src/qml/components/SnapshotSettings.qml)
  • File selection dialog
  • Snapshot verification status
  • Progress tracking and visualization

Key Design Decisions

  1. Non-blocking Implementation
  • Snapshot loading runs in separate QT thread
  • UI remains responsive during load
  1. Extensibility
  • Interface designed to accommodate future assumeutxo changes
  • Error handling framework in place

Testing Instructions

  1. Build
  2. Launch Bitcoin Core QML GUI
  3. Navigate to Connection Settings (after onboarding is done)
  4. Test with sample snapshot:
magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat
  1. Verify (see screenshots below):
    • "Snapshot Loaded" confirmation appears
    • Block clock shows ~50% progress
    • For errors, check debug.log for [snapshot] or [loadsnapshot] messages
Ubuntu 22.04 Screenshots

Screenshot 2024-10-11 100308
Screenshot 2024-10-11 100319
Load_UTXO_Snapshots_FileDialog
Screenshot 2024-11-13 144609
Screenshot 2024-11-13 144917
Screenshot 2024-11-13 145350

Expected Behavior

  • File selection dialog works
  • Success/failure state properly displayed
  • Node becomes usable while background validation continues

Future Work

  1. Error Handling
  • Add user-facing error messages
  • Implement recovery paths
  • Add snapshot validation UI once background validation has completed
  1. Integration
  • Coordinate with upstream assumeutxo changes
  • Add support for network-specific snapshots (testnet and mainnet)
  • Integrate with initial setup wizard

Notes for Reviewers

Snapshot Compatibility

  • Load option available after Onboarding in the Connections Settings page
  • Currently only works with signet UTXO snapshot
  • Limited to specific 160,000 height
  • Validates against chain parameters m_assumeutxo_data
    (the changes there can be discarded once synced since m_assumeutxo_data is already hardcoded)

feedback welcome on the approach and implementation details.

@D33r-Gee D33r-Gee marked this pull request as draft October 11, 2024 17:11
@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from cb409e3 to 670ca05 Compare October 15, 2024 17:43
@D33r-Gee
Copy link
Contributor Author

With cb409e3 rebased and got rid off the extra options_mode settings which are not relevant for this evaluation.

Also deleted the extra loadsnapshot.cpp file and instead added the load snapshot function in src/node/interfaces.cpp

@yashrajd
Copy link

Should I be getting a binary specific for this issue? I tried loading a signet snapshot into the one for #421 but was unable to...

@D33r-Gee
Copy link
Contributor Author

Should I be getting a binary specific for this issue?

Yes please download the artifact for your system here

I tried loading a signet snapshot into the one for #421 but was unable to...

That's correct behaviour, 421 is UI only and doesn't not have backend wiring

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 670ca05 to b77f7c1 Compare October 16, 2024 19:53
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Oct 16, 2024

With b77f7c1 minor tweaks to the src/node/interfaces.cpp snapshotload() function.

It mimics the rpc call loadtxoutset from bitcoin/bitcoin repository

Next will connect the loading to the progress bar

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from b77f7c1 to c59e557 Compare October 22, 2024 21:57
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Oct 22, 2024

with c59e557 Integrated the UI with the C++ backend, implementing a progress callback function to dynamically update the loading progress based on the number of coins processed.

Touched src/validation.h and src/validation.cpp to extract the coin loading progress.

Next: will update the "Loaded Snapshot" stack with the real info from the snapshot

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from c59e557 to e3f8c70 Compare October 25, 2024 20:42
@D33r-Gee
Copy link
Contributor Author

With e3f8c70 moved the load snapshot functionality from nodeModel to optionsModel since it make it easier to store the snapshot info and reduces redundant variable declarations.

Copy link
Member

@jarolrod jarolrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With e3f8c70 moved the load snapshot functionality from nodeModel to optionsModel since it make it easier to store the snapshot info and reduces redundant variable declarations.

e3f8c70 is a regression because optionsmodel is not an appropriate place for this, and we are now violating the role of settings.json to store the state of a process.

NodeModel and AppMode are more appropriate places for this information. But the entire approach here is still a concept NACK as we are performing unecessary changes to validation and chainparams.

@yashrajd
Copy link

yashrajd commented Oct 29, 2024

Ok I tested artefact#235 and functionality & UI seems to work.

Didn't see block clock so couldn't verify easily. Adding some UI feedback below:

assumeutxo-424-sc1 copy

  • fix list item width

assumeutxo-424-sc2 copy

assumeutxo-424-sc4 copy

  • fix list item width
  • fix icon size

assumeutxo-424-sc3 copy

  • blockHeight is accurate
  • the date in the copy above it seems right
  • didn't check hash accuracy
  • hash value is overflowing as you can see, I'd probably wrap it over 2-3 lines, perhaps add copy functionality

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from e3f8c70 to 90dc0dd Compare October 31, 2024 22:09
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Oct 31, 2024

with 90dc0dd rebased over main and addressed @jarolrod concerns by reverting back to just a minimal integration of the loading snapshot functionality.

@yashrajd I will address the UI issues next

As a follow up will investigate how to access the snapshot verification progress

Also looking into how to make the snapshot data persist and be available for display (i.e. snapshot block height, date, and merkle hash)

@D33r-Gee
Copy link
Contributor Author

But the entire approach here is still a concept NACK as we are performing unecessary changes to validation and chainparams.

For the chainparams the changes there are only temporary for evaluating and testing within this repo. I checked upstream and the signet snapshot has already been hardcoded so once our repo is synced the changes in chainparams.cpp can be discarded.

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 90dc0dd to 85592f5 Compare November 1, 2024 18:41
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Nov 1, 2024

with 85592f5 made the snapshot activation persist on node restart by making use of the interface with chain->hasAssumedValidChain

Also changed the green-check icon size to 30 pix

Next looking into how to extract the progress from the ActivateSnapshot process

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 85592f5 to 955c65d Compare November 4, 2024 22:39
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Nov 4, 2024

with 955c65d modified the snapshotLoad() function in src/node/interfaces.cpp and the initializeSnapshot() function in nodemodel.cpp to extract progress information using regular expressions.

This design decision aimed to keep snapshot loading mechanisms self-contained.

However, it looks like the use of std::stod is triggering some lint failures... will look into how to address that.

Perhaps the way to go is to use the handler in src/node/interfaces_ui ... will explore that path then update

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 955c65d to 0eba513 Compare November 8, 2024 18:35
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Nov 8, 2024

with 0eba513 reverted back to minimal snapshot load functionality.

Snapshot loading progress is still not were I want it to be so in the meantime replaced the progress bar with placeholder text.

Also updated the snapshot info display. once successfully loaded, and I'm using the hardcoded info in chainparams since if the snapshot has loaded successfully it means the info there has been validated.
This is make the info persistent and not rely on the snapshot.dat file (in case it gets deleted)

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 0eba513 to cd2ec69 Compare November 12, 2024 21:30
@D33r-Gee
Copy link
Contributor Author

With cd2ec69 added a section to keep track of snapshot loading, it is not ideal so will look for better ways to achieve it.

Since the core functionality is there will ask for "pre-reviews" before un-drafting

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch 2 times, most recently from 5d0756c to 262bb03 Compare November 13, 2024 21:23
@D33r-Gee
Copy link
Contributor Author

with 5d0756c updated SnapshotSettings.qml with comments documenting the loading process.

with 262bb03 updated nodemodel by adding the m_snapshot_loading bool so that the "Snapshot Loading" page persist if the user navigates away from it.

Next looking into how to finesse the snapshot loading progress...

@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Feb 3, 2025

With 68efab5 this PR is ready to transition from draft status.

The changes are separated into two distinct commits:

  1. Core functionality changes intended for upstream integration into bitcoin/bitcoin (bd0f120)

  2. QML-specific implementations and adaptations (68efab5)

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 68efab5 to 109cf42 Compare March 4, 2025 19:27
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Mar 4, 2025

109cf42 rebased over main

@D33r-Gee
Copy link
Contributor Author

with cad966e rebased over main.

Keeping this PR as draft until we get closer to the sync with upstream...

@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from cad966e to 6c25cd7 Compare April 24, 2025 20:55
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented Apr 24, 2025

with 6c25cd7 :

Will keep this as draft until we get closer to the sync with upstream

@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented May 2, 2025

4e43df9 updates the following:

  • Introduce isIBDComplete bool in qml/nodemodel
  • Update ConnectionsSettings.qml to reflect the change

Once the CI completes will undraft since we are not exactly sure when the sync with upstream will happen. And this feature will be demoed at upcoming events.

Feedback welcomed!

@D33r-Gee D33r-Gee marked this pull request as ready for review May 2, 2025 20:04
@D33r-Gee D33r-Gee changed the title [DRAFT] WIP QML Load Snapshot Signet (160,000 height) QML Load Snapshot Signet (160,000 height) May 2, 2025
@D33r-Gee D33r-Gee requested review from GBKS and johnny9 May 5, 2025 14:50
@D33r-Gee D33r-Gee requested a review from yashrajd May 7, 2025 14:14
@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from 4e43df9 to bbbbf42 Compare May 9, 2025 18:17
@D33r-Gee
Copy link
Contributor Author

D33r-Gee commented May 9, 2025

With bbbbf42 adds error page to handle when a invalid snapshot gets used, this is based on @GBKS feedback in #449

Feedback welcomed!

@hebasto
Copy link
Member

hebasto commented May 12, 2025

Based on #421.

This is confusing, given that #421 is closed.

D33r-Gee added 3 commits May 15, 2025 09:02
- This introduce the UI flow to load a AssumeUTXO snapshot
- It modifies the connection settings
- Adds a SnapshotSettings file, Icon, and modified progress
bar.
- Also it adds error page on snapshotloading failure
- Extend node interface with virtual functions for UTXO snapshot loading
- Add signal mechanism to monitor snapshot loading progress
- Include predefined signet UTXO dataset in chainparams for validation
- Add SnapshotQml class to encapsulate snapshot loading operations
- Integrate QML properties for UTXO snapshot management
- Wire up snapshot loading functionality with node interface
@D33r-Gee D33r-Gee force-pushed the qml-load-snapshot-signet branch from bbbbf42 to e02f382 Compare May 15, 2025 16:36
@D33r-Gee
Copy link
Contributor Author

e02f382 rebased over main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants