Build a Local-First X Research Companion with Gem Search

What Gem Search does

Gem Search is an open-source, local-first research companion for X. It adds an animated spider overlay to Chrome or Chromium, inspects posts rendered in the active tab, and sends small captures to a research engine running on your computer.

The local JEV detector deduplicates posts and groups recurring topics, project links, cashtags, valid Solana addresses, and emerging two-word phrases. A bounded crawler can follow public HTTP or HTTPS links, while the dashboard preserves sources, checks, gaps, activity, and exportable findings.

Gem Search is designed for investigation rather than automated trust. A lead, score, or clean-launch result does not prove that a project is safe, original, or likely to increase in value.

The extension reads the current page DOM. It is not an X firehose, does not interact with posts or accounts, and cannot monitor other applications or background tabs.

Key features

• Visible browser companion: The animated spider highlights the post it is inspecting and provides capture counters, pause controls, dismissal, and optional visible-tab auto-scroll.
• Local signal detection: JEV groups topics, detects tickers and Solana addresses, discovers repeated phrases, measures six-hour growth, and checks author diversity and repeated text.
• Evidence trail: Findings retain the posts, public links, checks, unknowns, and model reasons behind them.
• Bounded crawler: Public pages are cached for 30 minutes, and private-network targets are blocked.
• Durable local queue: Up to 200 captured posts can remain in the extension when the backend is unavailable and are forwarded after reconnection.
• Optional Grok review: Four role-specific reviewers analyze supplied evidence through the official xAI API when explicitly enabled.
• Local customization: You can define up to 40 English or Russian topic patterns in a local configuration file.
• Research export: The dashboard supports JSON export and browser-generated SVG discovery capsules.

Understand the architecture

The normal research flow is straightforward:

1. The extension observes rendered posts in the visible X tab.
2. Captures enter a persistent extension queue.
3. A paired localhost API sends them to the local engine.
4. JEV deduplicates and groups the signals.
5. The crawler retrieves bounded public sources.
6. Local evidence checks run before any optional model review.
7. The dashboard presents findings, sources, reasons, and unknowns.

The extension pairing token can submit captures and read scanner status. It cannot launch tokens, and the extension contains neither account-action code nor an xAI API key.

Prerequisites

Before installing Gem Search, prepare the following:

• Python 3.11 or newer.
• Chrome or another Chromium-based browser.
• macOS, Linux, or Windows through WSL2 for the backend.
• Git for cloning the repository.

Node.js 22 or newer is required only for the complete test suite and optional Solana module. The core research backend and unpacked browser extension do not need an npm build.

Install and start the local engine

Clone the repository, enter its directory, create a local environment file, and launch the backend:

git clone https://github.com/h100envy/gem-search.git
cd gem-search
cp .env.example .env
python3 app.py

Keep this terminal running. The dashboard should become available at http://127.0.0.1:8787.

Try the fictional demo first

Before connecting an X account, open http://127.0.0.1:8787/spider-demo. This page uses the same spider renderer over fictional cards, makes no API calls, and collects nothing.

The dashboard also has a separate demo scan that exercises the research pipeline with clearly marked synthetic projects.

Load and pair the Chrome extension

1. Open the local dashboard and go to Connections.
2. Copy the local pairing code.
3. Open chrome://extensions in Chrome or Chromium.
4. Enable Developer mode.
5. Select Load unpacked.
6. Choose the repository's extension/ directory.
7. Open the Gem Search toolbar popup.
8. Paste the pairing code and select Connect.

This pairing connects the extension to the local API. The research engine must remain active while you use the extension.

Run your first research session

1. Open an X feed, search page, or profile page.
2. Open the Gem Search extension popup.
3. Select Release spider on this tab.
4. Scroll normally while the spider inspects rendered posts in the visible viewport.
5. Watch the capture counter and pause or dismiss the spider whenever necessary.
6. Return to the local dashboard and open Discovery.
7. Inspect the grouped findings, source posts, linked pages, local checks, and explicitly unknown claims.
8. Export useful findings as JSON for later research.

Auto-scroll is optional. Background tabs do not collect posts or scroll automatically. If the local engine goes offline, the extension can retain up to 200 captures and forward them after the connection succeeds again.

Explore emerging signals

Gem Search can create separate leads for tickers, valid Solana contract addresses, and repeated phrases that do not belong to an existing topic.

Tickers

Each detected $CASHTAG can become a lead with its associated authors and posts. Major assets such as $BTC, $SOL, and $USDC, as well as price-like strings such as $100, are skipped. A lead can open a live X search.

Solana addresses

A candidate address is retained only when it decodes to a real 32-byte key. This reduces false positives from long random words. The resulting lead can link to the address on Solscan.

New narratives

An emerging phrase is a two-word phrase repeated by at least three different authors during the last six hours and found outside the configured topics. Overlapping wording from one sentence counts once.

Growth indicators

Each lead can show its mentions during the last six hours compared with the pace of the preceding 18 hours, the number of recent authors, and the time the spider first observed it. These measurements describe only your browsing sample, not all activity on X.

In Discovery, filter by signal type and sort by Fastest growing, Newest first, or Most authors. Open a lead to inspect its signal velocity and underlying posts.

Run the sample signal walkthrough

The repository includes a terminal demonstration that streams fictional posts through the real extension API and prints captures, engine events, leads, growth, and checks.

Start the engine in one terminal:

python3 app.py

In a second terminal, run the walkthrough:

python3 scripts/spider_demo.py

Wait about ten seconds for the spider loop, then open the dashboard. In Discovery, switch among ticker and new-narrative filters and sort by fastest growth.

Use the following command for a faster walkthrough:

python3 scripts/spider_demo.py --speed 2

For a silent sample load, use:

python3 scripts/sample_signals.py

The sample posts, handles, ticker, and address are invented and marked as samples.

Customize the topic detector

Copy the example topic file and edit the new local file:

cp topics.example.json topics.json

The file supports up to 40 topics expressed as case-insensitive regular expressions. Restart the backend after making changes.

If the file is malformed, Gem Search ignores it, logs a warning, and continues with the built-in topics. The custom topics.json file remains on your computer.

Enable four optional Grok perspectives

Gem Search works without Grok. To enable model-assisted review, add your own xAI settings to the local .env file:

GROK_ENABLED=1
XAI_API_KEY=your_local_api_key
GROK_MODEL=grok-4.7
GROK_DAILY_CALLS=12

Restart the engine after saving the file. The four local research seats ask different questions:

• Lookout: Is there a noteworthy signal in the observed sample?
• Maker: What product or technical evidence is actually present?
• Skeptic: What is contradictory, risky, or unverified?
• Runner: Is there enough evidence to investigate now?

A review starts only after the engine has observed at least four distinct authors. One complete review makes four API requests. With the default allowance of 12 requests per rolling 24 hours, no more than three complete reviews fit within the allowance, including failed calls.

Identical evidence and model inputs reuse cached results for 24 hours. Requests contain bounded excerpts and limit output to 700 tokens. The allowance controls request volume but does not guarantee a fixed bill; pricing depends on your xAI plan.

When Grok is enabled, collected excerpts are sent to api.x.ai for analysis. Keys remain in the backend. Without Grok, signal grouping and local checks continue to work entirely without model calls.

An unavailable model, exhausted allowance, invalid response, or invented citation cannot count as a passing result. Model output cannot execute code, browse independently, or control wallets. The README also notes that the adapter has mocked test coverage but that a real paid call was not validated in that checkout.

Create and share a discovery capsule

To turn a finding into a standalone visual record, open a project in Discovery and choose Discovery capsule. Select the Aurora or Daylight style, preview the card, and download the SVG.

The capsule can include mentions, observed authors, local checks, Grok status, up to three source addresses, and a UTC timestamp. It is generated entirely in the browser without an API call or automatic social post.

Credentials, query strings, and fragments are removed from displayed source addresses. Review the visible content before sharing, because missing evidence remains missing and demo findings remain marked as demos.

Use additional input sources

Gem Search is not limited to browser captures. It can import JSON captures according to the repository's documented data format, and the data/inbox/ directory can accept input from external collectors.

To enable the free Hacker News narrative scanner and inbox watcher, start the engine in autopilot mode:

python3 app.py --autopilot

Hacker News attention is labeled separately from X captures. Official X recent-search access remains optional and paid, with ENABLE_PAID_X=0 by default.

Advanced operating tips

• Start without paid services: Local grouping and evidence checks do not require an xAI key or paid X API.
• Inspect sources, not only summaries: Open the linked posts and public pages, then compare them with the local checks and unknown fields.
• Treat velocity as sample-specific: Rapid growth reflects the posts visible during your browsing session, not the entire network.
• Watch for coordinated behavior: Three coordinated accounts can manufacture an emerging phrase, so author diversity is a clue rather than proof.
• Reuse caches: Crawled pages are cached for 30 minutes, and identical Grok reviews are cached for 24 hours.
• Keep collection intentional: Use normal scrolling or opt into auto-scroll only when wanted, and pause the spider when the current page is irrelevant.
• Remember detector limits: Each pass keeps up to 10 tickers, 10 addresses, and five phrases, strongest first.
• Keep detected signals out of launch automation: Tickers, addresses, and emerging phrases never enter the launch queue, even when SPIDER_ALLOW_LAUNCH=1.

Hosted scanner and Telegram tools

The project also operates free tools at gemsearch.fun. These are distinct from the local browser research workflow and require no wallet connection for scanning.

• Scanner: Examines a Solana contract address for authorities, holder concentration, developer share, previous pump.fun launches, liquidity or curve progress, Dex profile information, and ticker clones.
• Web X-ray: Shows launch-block buyers, shared funding sources, and transfers among holders, with transaction links.
• Bundle Index: Classifies new pump.fun launches by the number of other launch-block buyers.
• Council record: Publishes stored verdicts and later outcome checks rather than hand-picked examples.
• Telegram bot: Accepts contract addresses and supports scanning, X-ray, watch, account tracking, and group commands.

A crew, cluster, or bundle represents an on-chain funding pattern, not proof that wallets belong to one identity. A high scan score indicates a cleaner observed launch, not a prediction of future performance.

Optional launch capabilities

Gem Search includes launch-related components, but they are separate from ordinary research. The hosted launch page can create a pump.fun coin from a browser wallet, while an earlier experimental local launcher remains opt-in and dry-run by default.

The hosted launch flow creates the mint key in the browser, stores image and metadata on IPFS, builds a transaction with the user as payer and owner, and submits it only when the signed transaction matches what the server built byte for byte. Gem Search states that it takes no platform fee and does not review or endorse launched coins.

For the optional local module, the documented ceiling is 0.025 SOL per attempt and five attempts per rolling 24 hours. Pump.fun is supported through the local PumpPortal executor, while another Solana platform requires a compatible HTTPS launch-bridge adapter. Other chains are not supported by the current SOL budget policy.

Run tests and package the extension

Install Node dependencies, run the Python and JavaScript tests, and package the extension with:

npm ci --ignore-scripts
python3 -m unittest discover -s tests -v
npm test
python3 scripts/package_extension.py

The packaging script writes dist/gem-search-extension.zip. The archive contains extension assets only. Extract it and load the extracted directory as an unpacked extension; it is not a Chrome Web Store package.

The automated tests cover validation, route exclusions, deduplication, narratives, Grok caching and allowances, failure handling, local-network blocking, queue recovery, and optional transaction guards. They do not guarantee compatibility with every future X layout, so real-browser behavior still requires manual testing.

Conclusion

Gem Search combines a visible browser overlay with a local signal detector, bounded crawler, persistent evidence trail, and optional Grok review. Start with the fictional demo, pair the unpacked extension, collect only the posts you intentionally view, and evaluate every finding through its sources and unknowns rather than treating a signal as a conclusion.

For implementation details, consult the repository's architecture documentation, privacy documentation, and contribution guide.