Identify Blockstack Users Within Gaia for Block Survey Migration

[andresgalante]

Block Survey is migrating away from Hiro's hosted Gaia Hub and needs a way to identify Blockstack users within Gaia. Specifically, we need to:

• Translate a user's domain name into their Gaia hub address.
• Access any APIs or methods that can help with user identification in this context.

Current Findings

• All Block Survey customers use the https://blocksurvey.io/ domain.
• Users have .btc or .id.blockstack domains.
• Some users do not have BNS names and are identified only via their wallet seed phrase.

Example User IDs

• compressstudio1.id.blockstack
• daniavorszky.id.blockstack
• david.id

Next Steps

• Investigate how to resolve user domain names to Gaia hub addresses.
• Determine the best approach for identifying users who lack BNS names.
• Estimate the effort needed:
  • Optimistically: 1-2 days for research and coding, followed by 1-2 days for DevOps and script execution.
  • Realistically: ~1.5 weeks.

cc: @zone117x

[andresgalante]

Here are the file names which will be there on BlockSurvey users Gaia:

mysurveysindex.json
blocksurvey_pgp_profile.json

[andresgalante]

The BlockSurvey team requires an increased rate limit to facilitate our migration away from Gaia. We can identify our requests using the header:

X-App-Identifier: blocksurvey

[raja-ilayaperumal]

Hi Team, This is a quick follow-up on the rate limit increment for BlockSurvey. It will help us to plan for a migration.

Any updates on this request?

[andresgalante]

Hey @raja-ilayaperumal, apologies for the delay on this.

We haven’t had a chance to get to it yet. I'll follow up with an update as soon as I can. I appreciate your patience!

[andresgalante]

Hi @raja-ilayaperumal

We’re having a hard time finding bandwidth on our side to take this on, so we wanted to check a couple of things that might help you move forward independently:

Does every Blocksurvey user bucket (Gaia hub address / directory) contain a clear indicator that it's associated with Blocksurvey?

For example, something like a blocksurvey-data.json file present in every relevant bucket. If that’s the case, it would make it easier for us to write a script internally to locate and potentially back up all Blocksurvey-related buckets.

Would your team be open to implementing some client-side migration logic yourselves?

For instance, you could add some client-side js that is ran when a blocksurvey page is loaded, and have it send something like the following payload to blocksurvey's servers:

• The user’s BNS ID
• Metadata
• Gaia hub bucket address
• List of Gaia URLs for Blocksurvey-related files

With that data collected, you could then build and run your own migration logic on the backend.

There’s no plug-and-play tooling for this, so your engineers will need to do some custom work either way.

Let us know what you think!

[raja-ilayaperumal]

@andresgalante Thanks for your reply.

I have gone through the two suggested approaches on GitHub.

We currently have a standard file name, "blocksurvey_pgp_profile.json," for every block survey user in Gaia. If you could get the user's list, that would also help us.

We plan to add the script to discover the Gaia-dependent user details. We will do it in the mid of May month.

[saralab]

Gaia Hub will be decommissioned May 30th, we will provide the data you need to save your data.
The team will reach out and update this issue with next steps and more details

[andresgalante]

from @raja-ilayaperumal on Discord

Hi @andy ,

I wanted to provide some context regarding the Gaia dependency across our three applications:

BlockSurvey – Initially, Gaia was used to build the decentralized survey platform. However, due to concerns over API rate limits, we transitioned away from it. The migration of older users' data to our storage is still pending, which aligns with the discussion you’re already aware of.
Owl.link – This decentralized bio-link tool is actively used by the community and also requires migration.
Ballot.gg – Our on-chain decentralized polling system currently supports Gaia but will need migration as well.

Given these dependencies, I’ve requested a revised timeline. Could you please bring this up with the board or team to consider and push the decommission timeline by June 30th?

Let me know if you’d like to discuss further.

[CharlieC3]

@andresgalante Any update on this?

[andresgalante]

Yes, I contacted Raja on Discord and sent him the link to the files on email.

He agreed to evaluate it and get back to me before the end of the week last week but I didn't hear back from him yet.

Forum post is out, and the plan is to disable Gaia on May 31st

[raja-ilayaperumal]

Hi @CharlieC3 and @andresgalante ,

Here is the quick update: We have got the BlockSurvey users' Gaia buckets from Andy. We plan to look into the Gaia migration this week and will get back to you by this weekend.

As I mentioned earlier regarding the Gaia dependency for our other two products—Owl.Link and Ballot. GG. We’re requesting an additional month of Gaia support. This will help us plan and execute the migration smoothly, without disrupting our customers or community usage. We need at least 60 days of notice from the date of decommissioning.

We’re also looking to obtain the user bucket files for Owl.Link and Ballot.GG for backup, similar to what we’ve done for BlockSurvey.org.

Here are the common file names for each product:

Ballot.GG – pollIndex.json
Owl.Link – owllink.json

Let us know the confirmation of the decommissioning date or support till 30th June 2025.

[andresgalante]

Hi @raja-ilayaperumal

We can extend Gaia support until June 15th, 2025 to help ensure the migration.
We'll also get you the user bucket data for Owl.Link (owllink.json) and Ballot.GG (pollIndex.json), similar to what was shared for BlockSurvey.

[0xc22b]

Hi @CharlieC3 and @andresgalante,

Can you also have a look at Brace.to and Justnote as well?
Thank you. 🙏

[raja-ilayaperumal]

Hi @andresgalante, Thanks for your confirmation. You can share the user's bucket details with my email address.

[andresgalante]

@raja-ilayaperumal I just sent you the email for Ballot and Owl

[andresgalante]

@0xc22b do you have a standard file name to identify the data?

[0xc22b]

@andresgalante

For Brace.to, a Gaia Hub bucket always contains a “links” folder, e.g., <bucket-address>/links/<...>.
For Justnote, a Gaia Hub bucket always contains a “notes” folder, e.g., <bucket-address>/notes/<...>.

🙏

[andresgalante]

@0xc22b What's an email where I can send you the buckets? or ping me on Discord

[0xc22b]

@andresgalante I've sent you my email on Discord. Thank you so much.

[raja-ilayaperumal]

Hi @andresgalante,

As we begin migrating from Gaia Storage to our cloud storage, we've encountered a key challenge:

How do we retrieve GaiaHubConfig details for connected accounts?
Currently, we rely on two API calls to hub.blockstack.org and gaia.blockstack.org to fetch Gaia config details. How can we determine a user's Gaia address if either of these services becomes unavailable?

Reference URLs:
https://hub.blockstack.org/hub_info
https://gaia.blockstack.org/hub/1LgMVdwPwaV46auY2MmxNYkWzQGXPpuxfv/profile.json

Note: We have all the users' buckets with the Gaia Address as reference. We need to know the user's Gaia address to allow the appropriate user to access/write the file.

Looking forward to your thoughts and suggestions.

[0xc22b]

@raja-ilayaperumal If I'm correct, a user's Gaia address derives from the user's app private key. Line 220 could help. In gaiaConfig, some info is from hubInfo, which is the same for all users, and some is derived from the user's app private key, which is distinct.

[raja-ilayaperumal]

Thank you for your response and the GitHub reference, @0xc22b. I am able to successfully derive the Gaia address from the user's app private key, and it works as expected.

One last challenge: We allowed users to create multiple accounts under a single secret key in the early stages. How can we retrieve all associated accounts for a given secret key or seed phrase without relying on Gaia Storage?

Looking forward to your input to get it sorted out.

[0xc22b]

@raja-ilayaperumal Use the deriveAccount function with an index to create an account at that index. For example, the first account is index 0, and the second is index 1. A walletConfig stored in Gaia tells how many accounts were created, so we can create accounts with an index to that number and let the user choose.

[raja-ilayaperumal]

@0xc22b Yes, I can retrieve my accounts using the deriveAccount function if the WalletConfig is accessible. I also found that account namespaces are retrieved from there. So, WalletConfig should be available from Gaia.

Will the WalletConfig be available after the Jun 15 Gaia decommission? If so, we are good. If not, what is another way to fetch these wallet and namespace details?

[0xc22b]

@raja-ilayaperumal The deriveAccount function doesn't need to depend on a walletConfig. It depends on a secret key and account index starting from zero. You can pre-generate 10 first accounts (index 0 to 9) and let a user choose. Or you display the first account and have a button to derive the next one.

You can also copy walletConfig data to your Gaia. A walletConfig bucket address is derived similarly to the app data bucket address, e.g., use wallet.configPrivateKey instead of appPrivateKey. But you need to copy all wallet-config.json from Hiro-hosted Gaia to yours.

I don't know about namespaces and WalletConfig availability after Jun 15. I'll leave them to the Hiro team (@andresgalante ) to answer.

[raja-ilayaperumal]

@0xc22b Yes, we will have a dependency to fetch the number of existing accounts and namespaces alone. Thanks for your support and help with the migration.

@andresgalante, you can confirm the walletConfig availability with us after Jun 15.