FurryCoder

Feopack: Mini Rspack

Sun, 03 May 2026 00:00:00 GMT

Why did I build it?

Feopack started as a learning project.

Not because the JavaScript ecosystem needed another production bundler. It definitely did not wake up one morning asking for my tiny Rust experiment. I built it because I wanted to understand what modern bundlers like Rspack are doing under the surface.

Rspack is much faster than webpack because it moves many heavy tasks into the native layer. This helps avoid the bottleneck of JavaScript's single-threaded execution model and reduces the overhead of communication inside the build pipeline.

flowchart LR
  Entry["【Entry Points】<br/>main.js / another entry"]

  subgraph MAKE["Make"]
    direction TB
    Queue["【Module Worker Queue】<br/>parallel module jobs"]
    Build["【Build & Analyze】<br/>resolve deps / run loaders / parse AST"]
    ModuleGraph["【ModuleGraph】<br/>modules + dependency edges"]
    Queue --> Build
    Build -- "discovered deps" --> Queue
    Build --> ModuleGraph
  end

  subgraph SEAL["Seal"]
    direction TB
    Optimize["【Optimization Analysis】<br/>used exports / side effects"]
    ChunkGraph["【ChunkGraph】<br/>code splitting + chunk groups"]
    Ids["【ID Generation】<br/>module ids + chunk ids"]
    Codegen["【Code Generation】<br/>tree shaking + final module code"]
    Runtime["【Runtime Modules】<br/>chunk loading / helpers"]
    Assets["【Chunk Assets】<br/>runtime chunks + normal chunks"]
    Optimize --> ChunkGraph
    ChunkGraph --> Ids
    Ids --> Codegen
    Codegen --> Assets
    Runtime --> Assets
  end

  subgraph EMIT["Emit"]
    direction TB
    Output["【Emit Assets】<br/>write files to output"]
  end

  Entry --> Queue
  ModuleGraph --> Optimize
  ChunkGraph --> Runtime
  Assets --> Output

I guess someone once tried to rebuild the whole webpack ecosystem in C++, then do something Rspack-like, and got lost somewhere halfway through the maze. Maybe webpack is simply too huge, and too deeply married to Tapable, to be casually rebuilt from scratch.

And somehow, Rspack actually did it.

Of course, it also had to carry a lot of luggage from the webpack era. Compatibility is not free, and when you inherit that much history, perfect performance is probably not included in the box.

But that was exactly what made it interesting to me. It suggested that there might still be room to push things further, to rethink a few pieces, and maybe, someday, to take part in building something in that direction. That curiosity is basically why I built feopack.

Anyway, back to feopack itself. Let's see what I actually did.

This post follows the order in which the project grew. It is less of a complete tutorial and more of a development journal: what I implemented, what I learned, and where the complexity started to show up and politely ruin my afternoon.

BTW, the name feopack is part of the joke. Rust can mean iron oxide, and FeO is also an iron oxide, just not the fully rusted one. So feopack is a small, not-fully-rusted, Rspack-like experiment.

1. Starting from the JavaScript Wrapper

The first meaningful layer was not Rust. It was the JavaScript-facing API.

That might sound strange for a Rust-based bundler, but it makes sense. Most users do not interact with the compiler core directly. They interact with a function, a config file, a CLI command, and a Compiler object.

flowchart LR
  Config[User Config] --> Wrapper[TypeScript Wrapper]
  Wrapper --> Adapter[Raw Options Adapter]
  Adapter --> NAPI[NAPI Binding]
  NAPI --> Core[Rust Compiler Core]

So before worrying about module graphs and chunk graphs, I needed to make this JavaScript-to-native handoff feel boring in the best possible way.

In feopack, the public API begins with something like this. Nothing fancy, which is exactly the point:

import feopack from "feopack";

const compiler = feopack({
  context: process.cwd(),
  entry: "./src/index.js",
  output: {
    path: "./dist",
    filename: "main.js",
  },
});

compiler.run();

Internally, this creates a Compiler instance. The TypeScript wrapper owns the user-facing shape of the API, while the Rust side owns the actual compilation work.

But a friendly JavaScript API alone is not enough. It still needs a bridge into the native layer, which brings us to the binding.

2. Making the Boundary Explicit with NAPI

Luckily, napi-rs does most of the heavy lifting needed to build that bridge.

The TypeScript wrapper only needs to lazily create a native compiler instance from @feopack/binding. On the Rust side, that native class receives RawOptions, converts them into internal compilation options, and exposes an async build() method.

In the bigger Rspack-shaped picture, this is the highlighted Configuration & Binding slice: the place where user-facing configuration turns into something the native compiler can actually work with.

graph TD
  subgraph Entrances[Entrances]
    direction TB
    CLI["@rspack/cli<br/>Command Line Interface"]
    DevServer["@rspack/dev-server<br/>Development Server"]
    CoreJS["@rspack/core<br/>JavaScript API"]
  end

  subgraph PluginEcosystem[Plugin Ecosystem]
    direction TB
    JSPlugins["JavaScript Plugins<br/>Tapable Hook System"]
    RustPlugins["Rust Plugins<br/>Native Performance"]
    BuiltInPlugins["Built-in Plugins<br/>Core Functionality"]
  end

  subgraph ConfigBinding[Configuration & Binding]
    direction TB
    ConfigSys["Configuration System<br/>rspack.config.js processing"]
    BindingBridge["@rspack/binding<br/>N-API Bridge"]
  end

  subgraph RustCore[Rust Core Engine]
    direction TB
    RustCompiler["rspack_core::Compiler<br/>Build Orchestration"]
    RustCompilation["rspack_core::Compilation<br/>Single Build Instance"]
    ModuleGraph["ModuleGraph<br/>Dependency Relationships"]
    ChunkGraph["ChunkGraph<br/>Output Structure"]
  end

  CLI ==> CoreJS
  DevServer ==> CoreJS
  CoreJS --> ConfigSys
  CoreJS -.-> BindingBridge
  ConfigSys --> BindingBridge
  BindingBridge ==> RustCompiler
  JSPlugins ==> BindingBridge
  RustPlugins ==> RustCompiler
  BuiltInPlugins ==> RustCompiler
  RustCompiler ==> RustCompilation
  RustCompilation ==> ModuleGraph
  RustCompilation ==> ChunkGraph

  style ConfigBinding fill:#fff3bf,stroke:#f08c00,stroke-width:3px,color:#000
  style ConfigSys fill:#ffe066,stroke:#f08c00,color:#000
  style BindingBridge fill:#ffe066,stroke:#f08c00,color:#000

The so-called binding is essentially a platform-specific native library loaded from Node.js. It is the small door JavaScript uses to knock on Rust's very serious office.

This boundary forced me to make the data structures explicit. JavaScript objects can be flexible and loose. Rust compiler options cannot be quite that casual. The binding layer becomes the place where the two worlds agree on a contract, preferably before either side starts throwing confusing errors at the other.

While reading Rspack's binding layer, I found it useful to think about three kinds of cross-language traffic.

First, JavaScript can hold a reference to Rust state. This is the normal Compiler story: Rust owns the real compiler instance, while JavaScript receives an object that can call methods like build(). In a webpack-compatible world, this also matters because JavaScript plugins may need to observe or interact with runtime compiler data. JavaScript is not rebuilding the compiler; it is holding the remote control.

Second, Rust can hold a reference back to JavaScript. This is needed when an object passed from JS contains callbacks, factories, or other JS-owned behavior. Rust should not try to “interpret” that JavaScript by itself. Instead, it keeps a safe reference to the JS value and asks the JS runtime to execute it when needed. In Rspack, types like ThreadsafeJsValueRef and ThreadsafeFunction exist for this kind of situation: Rust can call back into JS without pretending V8 is just another Rust crate.

Third, sometimes the best answer is no reference at all: just copy the data. Configuration is a good example. A config object is usually expected to be stable during a build, so the binding layer can convert the incoming JS object into Rust-side options. This is where RawOptions becomes CompilerOptions, and the real work is less glamorous than it sounds: flatten JavaScript's dynamic shapes into Rust's stricter types.

For example, a JavaScript option might allow multiple shapes, while Rust wants an explicit enum:

let pathinfo = match value.pathinfo {
  Either::A(value) => PathInfo::Bool(value),
  Either::B(value) => PathInfo::String(value),
};

Some options can even be either a plain value or a JavaScript callback. The Rust type has to describe that honestly:

pub struct JsFilename<F = ThreadsafeFunction<(JsPathData, Option<JsAssetInfo>), String>>(
  Either<String, F>,
);

That made the binding layer feel less like glue code and more like a customs office. Objects arrive from JavaScript with flexible passports, and Rust politely asks everyone to fill out the correct forms before entering the compiler core.

I also had a few ideas along the way, such as passing a Node-side file system or resolver factory into Rust.

I did not implement them in feopack at this stage, but they were useful reminders of where the boundary could grow later.

I also started another library project named @rush-fs/core, which is one small experiment in that direction. Hopefully, I will eventually find enough time outside work to finish all these tiny side quests.

After the wrapper and binding skeleton existed, I added a CLI and a small playground. A bundler without a way to run examples is mostly just a collection of confident lies.

3. Adding a CLI and Playground to Show My Love for TDD

On the one hand, I believe everyone loves TDD, because it gives us a nice, repeatable way to check whether our logic actually works.

On the other hand, there is one tiny problem: I am sometimes too lazy to come up with good test cases from scratch. A tragic flaw, I know.

So I borrowed some cases from Rspack's source code, made a few small changes, and wired them up so they could be triggered with commands like pnpm test chunk/basic. The goal was simple: run a case, generate assets, and stare at the output until the bundler confessed what it was doing.

To be honest, I no longer remember exactly where I found those cases. I later checked the Rspack and Webpack repositories again and somehow could not find them. Maybe they moved. Maybe I imagined them.

Either way, the playground survived, which is what really matters.

With that in place, I could finally start building the actual compiler pipeline.

4. Building the Compilation Lifecycle

As mentioned earlier, the core introduced a Compiler and a Compilation.

Compiler owns the high-level build process.
Compilation owns the state of one build: options, module graph, chunk graph, and generated assets. I think of it as the working memory for a single build.

In feopack, the outer calls are mostly wrappers around the real bundler core. The core lifecycle follows the familiar three-step shape: make, seal, and emit.

flowchart LR
  subgraph BundlerCore[Compilation]
    direction LR
    Make["Make"] --> Seal["Seal"] --> Emit["Emit"]
  end

  Run["compiler.run()"] --> Make

This is close to how many bundlers organize their work. The names may vary, and production bundlers have far more hooks and intermediate states, but the main rhythm is usually the same.

Make is where modules are discovered and the module graph starts to grow.
Seal is where that graph becomes chunks and generated assets.
Emit writes the final files to disk.

Some stages, such as CompileDone and MakeDone, were simplified away. That was intentional. I wanted to keep the lifecycle visible without pretending the hook system existed yet.

Yes, the hook system is still unfinished. But someday, maybe I will get to it. Probably right after finishing all the other “small” ideas.

5. From Source Files to a Module Graph

From this point on, the project became a series of data-structure transformations.

A bundler starts with files on disk, but files are not a very useful shape for a compiler. The first job of make() is to turn source files into a graph: each module becomes a node, and each import becomes an edge to another module.

In feopack, the simplified Module looked like this:

pub struct Module {
  pub id: String,
  pub dependencies: Vec<String>,
}

Storing dependency strings directly is not ideal. A more serious bundler would usually store references, dependency objects, or richer edges, especially if it wants incremental builds. But for this stage, the goal was to make the shape visible: “this module depends on those modules.”

The rough transformation was:

Take the chunk/basic playground case as an example. The entry file is almost embarrassingly small, which is exactly why it is useful:

// src/index.js
import title, { num, plusNum } from "./app.js";

title("Hello FeOPack");
console.log(num);
plusNum();
console.log(num);

flowchart LR
  Source["【Source File】"] --> Imports["【Import Request】<br/>request = './app.js'"]
  Imports --> Resolved["【Resolved ID】<br/>.../src/app.js"]

  subgraph ModuleGraph["ModuleGraph"]
    direction TB
    IndexModule["【Module】: index.js<br/>id: .../src/index.js<br/>deps:['./app.js']"]
    AppModule["【Module】: app.js<br/>id: .../src/app.js<br/>deps: []"]
    IndexModule --> AppModule
  end

  Resolved --> AppModule

The make() phase uses SWC to parse each module and read its import declarations.

A small VecDeque queue then keeps track of which resolved files still need to be processed. In real Rspack, this would involve EntryDependency, a ModuleFactory, and a task loop for parallel scheduling. feopack keeps only the minimal version: parse with SWC, resolve the discovered imports, push the next module paths into the queue, and repeat.

queue.push_back(entry_path);

while let Some(module_path) = queue.pop_front() {
  if visited.contains(&module_path) {
    continue;
  }

  visited.insert(module_path.clone());
  // read source, parse with SWC, resolve imports, queue dependencies
}

Module identity is not a detail here. In the playground case, ./app.js is only the request written in src/index.js; the compiler still needs a stable internal ID, usually a normalized resolved path like .../src/app.js.

Without that normalization, two equivalent paths can accidentally become two different modules, and the graph stops being a graph of the program.

It becomes a graph of spelling choices.

6. From JavaScript Syntax to Import Records

The module graph needs dependencies, but JavaScript source code does not hand them over as a neat list. It gives you syntax.

That is where SWC enters the design. Writing a JavaScript parser would be a different project. For a bundler, the more relevant question is how parser output becomes compiler state. SWC provides the AST; feopack decides which parts of that AST become import records, graph edges, and eventually runtime code.

Take the same line from src/index.js:

import title, { num, plusNum } from "./app.js";

For this import line, the important conversion is not the full AST shape. The useful data flow is how this declaration becomes records the compiler can use:

flowchart LR
  ImportDecl["【Import】<br/>import title, { num, plusNum } from './app.js'"]

  subgraph RawRecords["RawImportRecord[]"]
    direction TB
    RawDefault["【Raw】default import<br/>local: title<br/>imported: default<br/>request: './app.js'"]
    RawNum["【Raw】named import<br/>local: num<br/>imported: num<br/>request: './app.js'"]
    RawPlus["【Raw】named import<br/>local: plusNum<br/>imported: plusNum<br/>request: './app.js'"]
  end

  subgraph ResolvedRecords["ResolvedImportRecord[]"]
    direction TB
    ResolvedDefault["【Resolved】 title<br/>request: './app.js'<br/>module_id: .../src/app.js"]
    ResolvedNum["【Resolved】 num<br/>request: './app.js'<br/>module_id: .../src/app.js"]
    ResolvedPlus["【Resolved】 plusNum<br/>request: './app.js'<br/>module_id: .../src/app.js"]
  end

  ImportDecl --> RawRecords
  RawDefault --> ResolvedDefault
  RawNum --> ResolvedNum
  RawPlus --> ResolvedPlus

The raw import record keeps what the source code says:

pub struct RawImportRecord {
  pub local: String,
  pub imported: String,
  pub request: String,
}

Then resolve_imports() turns that into a resolved record by adding the real module ID:

pub struct ResolvedImportRecord {
  pub local: String,
  pub imported: String,
  pub request: String,
  pub module_id: String,
}

That extra module_id is where syntax becomes compiler state. In the diagram, request keeps the original source-level value, ./app.js, while module_id points to the resolved module, .../src/app.js. Keeping both fields avoids mixing two different concepts: the user's module request and the compiler's canonical module identity.

The imports are also stored in a Vec, not a map. The reason is not performance; it is semantics. Import declarations have source order, and source order can matter once transformations, side effects, or generated helper statements enter the picture. A map would make lookup convenient, but it would quietly discard ordering information. That is exactly the kind of “small simplification” that later becomes a semantic bug.

7. From Module Graph to Chunk Graph

Once the module graph existed, the next question was: how should these modules be packaged?

That is the job of the chunk graph. A module graph describes relationships in the source program. A chunk graph describes the runtime packaging plan. They are related, but they answer different questions.

In feopack, the chunk data structure was intentionally tiny:

pub struct Chunk {
  pub id: String,
  pub module_ids: Vec<String>,
}

pub struct ChunkGraph {
  pub chunks: Vec<Chunk>,
}

The transformation is deliberately simple:

flowchart LR
  subgraph ModuleGraph["ModuleGraph"]
    direction TB
    IndexModule["【Module】: index.js<br/>id: .../src/index.js"]
    AppModule["【Module】: app.js<br/>id: .../src/app.js"]
    IndexModule --> AppModule
  end

  ModuleGraph --> CreateChunk["【Create Chunk】"]
  Strategy["【Chunking Strategy】<br/>put all modules into main"] --> CreateChunk

  subgraph ChunkGraph["ChunkGraph"]
    direction TB
    MainChunk["【Chunk】: main<br/>module_ids:<br/>.../src/index.js<br/>.../src/app.js"]
  end

  CreateChunk --> MainChunk

In a real bundler, this step would involve grouping strategies.

But I didn't complete it, the current version puts every module into one chunk. That means no advanced code splitting, no async chunk loading, no CSS ordering, and no optimization pass.

hope that one day I will finish it

let chunk = Chunk {
  id: "main".to_string(),
  module_ids,
};

self.chunk_graph.chunks.push(chunk);

Even in this simplified version, ChunkGraph has a different responsibility from ModuleGraph. ModuleGraph is about program structure: who imports whom. ChunkGraph is about delivery: what files will the runtime load, and which modules live inside them.

So the data shape moved one step forward:

source files -> modules -> module graph -> chunk graph

The bundler was still not “smart,” but the representation had moved from source units to runtime units.

8. From Codegen Modules to Generated Assets

The last transformation in this part is where a chunk becomes an output asset.

The important detail is that a Chunk does not contain generated JavaScript yet. In this version, it only contains module_ids: the list of modules that should be rendered together. Code generation then walks those IDs, consumes the module source cached during make(), rewrites its import/export syntax, and stores the result as a temporary CodegenModule:

struct CodegenModule {
  id: String,
  source: String,
}

This structure is a bridge between the compiler world and the runtime world. It no longer represents the original file exactly. It represents a module after its imports and exports have been rewritten into something the bundle runtime can execute.

During code generation, two pieces of compiler state meet: the chunk tells render_chunk() which modules belong to the output, and CodegenModule[] provides the transformed source for those modules.

flowchart LR
  subgraph ChunkGraph["ChunkGraph"]
    direction TB
    MainChunk["【Chunk】: main<br/>module_ids:<br/>.../src/index.js<br/>.../src/app.js"]
  end

  subgraph CodegenModules["CodegenModule[]"]
    direction TB
    IndexCodegen["【CodegenModule】: index.js<br/>id: .../src/index.js<br/>source: transformed JS"]
    AppCodegen["【CodegenModule】: app.js<br/>id: .../src/app.js<br/>source: transformed JS"]
  end

  MainChunk --> RenderChunk["【render_chunk】<br/>choose module order from chunk"]
  CodegenModules --> RenderChunk
  RenderChunk --> Runtime["【Runtime Module Table】<br/>__feopack_modules__"]
  Runtime --> Asset["【GeneratedAsset】<br/>filename: main.js"]

The final asset structure is simple:

pub struct GeneratedAsset {
  pub filename: String,
  pub source: String,
}

The important part is that render_chunk() turns a list of CodegenModules into a tiny module system:

const __feopack_modules__ = {
  "entry.js": (__feopack_module__, __feopack_exports__, __feopack_import__) => {
    // transformed module code
  },
};

const __feopack_cache__ = {};

function __feopack_import__(id) {
  if (__feopack_cache__[id]) {
    return __feopack_cache__[id].exports;
  }

  const __feopack_module__ = { exports: {} };
  __feopack_cache__[id] = __feopack_module__;
  __feopack_modules__[id](
    __feopack_module__,
    __feopack_module__.exports,
    __feopack_import__,
  );
  return __feopack_module__.exports;
}

The transition matters very much.

Once imports and exports enter the picture, a bundle is not just concatenated JavaScript. It is a small module system pretending to be a file.

This part gradually moved from a CommonJS-style idea toward something closer to ESM-style runtime behavior.

At first, I built the CJS-style version based on my webpack experience, but it produced incorrect bundles in a few cases.

So I started moving it toward an ESM-style runtime model, which fits modern module semantics better.

One rough edge remains in this implementation: it still creates fresh SwcCompiler instances in a few places. That is not the most elegant architecture. A more mature version would think harder about compiler ownership, shared source maps, and concurrency boundaries.

But for a small bundler, this tradeoff kept the pipeline explicit enough to reason about.

At this point, the chunk has stopped being only a packaging plan. It has become an in-memory GeneratedAsset, with a runtime table, transformed module functions, and an entry call. The later emit_assets() step is responsible for writing that asset to disk.

9. Where the Runtime Semantics Start to Bite

By the time GeneratedAsset exists, the main build path has already reached emit-ready output. What remains interesting is not another pipeline stage, but the semantic cost hidden inside code generation.

The pipeline sounds clean: source code becomes an AST, the AST becomes a transformed AST, and the transformed AST becomes JavaScript again. In practice, this is where earlier abstraction choices start to matter. The implementation has to decide how imports become runtime reads, how exports stay live, and how generated helpers fit into the module function.

There was also friction around SWC codegen APIs and ecosystem version compatibility. Emitting JavaScript from an AST required understanding Emitter, JsWriter, SourceMap, and how errors should be mapped back into feopack's own error type.

Import and export semantics were the real rabbit hole. feopack gradually added support for:

default imports
named imports
exported variables
exported functions
named export declarations
dynamic export definitions on the generated exports object

This required transforming import declarations into namespace objects and rewriting imported bindings. For example, a named import needs to become a property access on the imported module namespace. That is the role of the binding rewriter: find identifiers that came from imports and rewrite them to the generated namespace access.

The current implementation leaves many cases unsupported:

namespace imports are not supported yet
anonymous export default function is not supported
export ... from re-exports are not supported
destructuring export declarations are not supported
many non-JavaScript assets are out of scope

These limitations are useful because they mark the actual boundary of the implementation. Every JavaScript syntax form expands the bundler's semantic surface. Supporting import title, { num, plusNum } from "./app.js" is not just parsing a string. It affects graph resolution, AST transformation, runtime export definition, and generated code behavior.

10. Putting the Build Path Together

The useful way to look at this milestone is not as a feature checklist, but as a sequence of data structures. Each step gives the next step a more precise shape to work with.

flowchart LR
  Config["【User Config】<br/>entry / output / mode"] --> Compilation["【Compilation】<br/>build state"]

  subgraph Make["Make"]
    direction TB
    ModuleGraph["【ModuleGraph】<br/>source files -> modules + dependencies"]
  end

  subgraph Seal["Seal"]
    direction TB
    ChunkGraph["【ChunkGraph】<br/>chunk -> module_ids"]
    Runtime["【Runtime Table】<br/>chunk + transformed modules"]
    Asset["【GeneratedAsset】<br/>in-memory main.js"]
    ChunkGraph --> Runtime --> Asset
  end

  subgraph Emit["Emit"]
    direction TB
    Disk["【Output File】<br/>dist/main.js"]
  end

  Compilation --> ModuleGraph
  ModuleGraph --> ChunkGraph
  Asset --> Disk

This complete path is the part feopack currently implements: configuration enters from JavaScript, Rust builds a Compilation, make() constructs the module graph, seal() groups modules into a chunk and generates an in-memory GeneratedAsset, and emit_assets() writes it to disk.

Of course, this is still far from a real bundler. The obvious missing pieces are a plugin system, a more complete loader pipeline, CSS and asset modules, HMR, source maps, tree shaking, advanced chunk splitting, and much more complete ESM compatibility. If I continue working on feopack, these are probably the directions I will pick up one by one.

11. Why This Still Matters in the Agent Era

feopack is not production-ready, and it is not trying to be. That was never the point.

We are in a moment where the industry is full of new tools, new workflows, new model capabilities, and new reasons to feel behind. I pay attention to those things too. It would be strange not to. But the more noise there is, the more important it becomes to keep some independent technical judgment.

For me, that judgment cannot come only from reading announcements, watching demos, or collecting opinions from other people. It has to be grounded in the ability to take a system apart, rebuild a small version of it, and understand where the real complexity lives.

That is why writing feopack mattered. Not because the world needs another bundler, but because implementing even a small path through a bundler forces the abstractions to become concrete: config, binding, compilation, module graph, chunk graph, runtime table, asset, emit. After that, “bundler architecture” is no longer just a phrase. It has weight.

Writing about it is part of the same process. It is a way to leave evidence that I did not only skim the surface. I actually followed the data structures, hit the awkward edges, and formed my own understanding.

Or, to keep the original Chinese line that says this better than I can:

批五岳之图以为知山，不如樵夫之一足；谈沧溟之广以为知海，不如估客之一瞥；疏八珍之谱以为知味，不如庖丁之一啜。及之而后知，履之而后艰，乌有不行而能知者乎？

A Sui Chain TX Analyzer Based on NebulaGraph

Sun, 29 Jun 2025 00:00:00 GMT

Original post: DEV Community

Project Code: GitHub - CoderSerio/Sui-Nebula-Analyzer

I. Project Background and Challenges

As blockchain technology advances, on-chain data is exploding. Extracting valuable insights from this deluge and uncovering complex relationships is crucial for ensuring transaction compliance.

Yet, raw blockchain data is often fragmented and unstructured, making analysis difficult. Blockchains like Sui, with their object-based model, differ from traditional account-based ones, posing new data analysis challenges.

In blockchain relationship analysis, traditional relational databases have many limitations:

Poor performance for multi-hop queries: In complex transaction networks, multi-hop queries are needed. But SQL's JOIN operations struggle with large-scale, multi-level networks, failing to meet real-time analysis needs.
Difficulty modeling dynamic relationships: Relationships between wallet addresses on the blockchain change dynamically. Traditional relational databases can't easily model and adapt to these ever-changing relationships.
Limited graph algorithm support: Detecting abnormal on-chain behaviors, identifying money laundering patterns, or conducting community analyses requires graph algorithms like PageRank and community detection. Traditional databases lack native graph computing capabilities, requiring data export to specialized platforms, increasing complexity and latency.

Graph databases, with their natural graph structure storage and query abilities, efficiently handle complex relationship data and natively support various graph algorithms, making them ideal for blockchain data analysis.

So, I built a transaction relationship analysis platform using the NebulaGraph database to provide a solution for these problems.

II. Why Choose NebulaGraph?

NebulaGraph is a high-performance, distributed graph database whose design philosophy aligns with blockchain data analysis requirements. It can:

Complex relationship queries: It natively supports multi-hop queries, path analysis, and cycle detection via its graph query language (nGQL), enabling the discovery of deep relationships within vast on-chain data.
High-performance graph computing: It has built-in or easily integrable graph algorithms like PageRank, community detection, and centrality, providing robust support for advanced analysis.
Real-time analysis: It can respond to complex relationship queries in sub-seconds, meeting the real-time analysis demands of blockchain data.
Flexible modeling: The graph model naturally adapts to the dynamic nature of blockchain data, making it easy to model and extend entities like wallets, transactions, and objects, as well as their complex relationships.

III. System Architecture and Data Model

3.1 Overview of the Architecture

The project's backend services consist of two layers: Next.js Server and Gateway Server:

Frontend Application (Next.js Application): The user interface responsible for data visualization and user interaction.
API Layer (Next.js API Routes): Provides application-specific API interfaces, handles business logic, and coordinates with the Gateway Server for database operations. For instance, /api/execute for query execution, /api/data-collection for blockchain data ingestion, and /api/debug-db for database debugging.
Gateway Server (Express App): Since Nebula's HTTP API is provided by Nebula Gateway, and the Nebula Gateway SDK is not very compatible with Next.js Server, a pure Node.js service is used here to connect and access Nebula Gateway.
NebulaGraph Database: Stores and manages Sui on-chain data, offering high-performance graph query and computing capabilities.

3.2 Data Collection and Processing

Sui-Nebula-Analyzer enables automated on-chain data collection, transforming raw Sui blockchain data into structured data suitable for storage in a graph database. The collection process involves obtaining transactions, events, and object information from the Sui chain and mapping them to vertices and edges in NebulaGraph.

3.3 NebulaGraph Data Model

To efficiently store and query Sui on-chain data, the project has designed specific graph spaces, tags, and edge types in NebulaGraph.

Graph Space Management:

DROP SPACE IF EXISTS sui_analysis;
CREATE SPACE IF NOT EXISTS sui_analysis (partition_num = 10, replica_factor = 1, vid_type = FIXED_STRING(64));

Tag Definitions:

The wallet tag represents wallet addresses on the Sui chain, with the following properties:

CREATE TAG IF NOT EXISTS wallet (
  address string NOT NULL,
  first_seen datetime,
  last_seen datetime,
  transaction_count int DEFAULT 0,
  total_amount double DEFAULT 0.0,
  is_contract bool DEFAULT false,
  sui_balance double DEFAULT 0.0,
  owned_objects_count int DEFAULT 0,
  last_activity datetime
);

Edge Type Definitions:

The transaction edge type represents transactions between wallets, with the following properties:

CREATE EDGE IF NOT EXISTS transaction (
  amount double NOT NULL,
  tx_timestamp datetime NOT NULL,
  tx_hash string NOT NULL,
  gas_used int DEFAULT 0,
  success bool DEFAULT true,
  transaction_type string DEFAULT 'unknown'
);

The related_to edge type represents relationships between wallets, which are derived from analysis algorithms, with the following properties:

CREATE EDGE IF NOT EXISTS related_to (
  relationship_score double NOT NULL,
  common_transactions int DEFAULT 0,
  total_amount double DEFAULT 0.0,
  first_interaction datetime,
  last_interaction datetime,
  relationship_type string DEFAULT "unknown",
  avg_gas_used double DEFAULT 0.0
);

This data model design fully utilizes NebulaGraph's graph capabilities, enabling efficient storage and querying of complex on-chain relationships and laying the foundation for in-depth analysis.

IV. Core Features and Applications

Sui-Nebula-Analyzer extensively uses NebulaGraph's graph query capabilities across its functional modules. Below are practical application examples demonstrating efficient on-chain data analysis using nGQL.

4.1 Statistical Analysis

The project provides queries for basic on-chain statistics, helping users quickly understand the overall picture of the Sui network:

Counting wallets:

USE sui_analysis;
MATCH (n:wallet) RETURN count(n) as count;

Counting transactions:

USE sui_analysis;
MATCH ()-[e:transaction]->() RETURN count(e) as count;

Counting relationships:

USE sui_analysis;
MATCH ()-[r:related_to]->() RETURN count(r) as count;

4.2 Relationship Queries and Analysis

This is one of the core functions of Sui-Nebula-Analyzer. Through nGQL's powerful graph traversal and pattern matching capabilities, it uncovers complex relationships between wallets.

Basic mode: Querying relationships

This query retrieves precomputed relationships between wallets, usually based on the related_to edge type and its relationship_score attribute.

USE sui_analysis;
MATCH (a:wallet)-[r:related_to]-(b:wallet)
WHERE r.relationship_score >= 0.1
RETURN a.wallet.address AS addr1,
       b.wallet.address AS addr2,
       r.relationship_score AS score,
       r.common_transactions AS common_tx,
       r.total_amount AS amount
LIMIT 50;

Pro mode: Querying enhanced fields

Building on the basic query, this returns more detailed relationship information, such as average Gas consumption, first and last interaction times, relationship types, and wallet balances and object counts.

USE sui_analysis;
MATCH (a:wallet)-[r:related_to]-(b:wallet)
WHERE r.relationship_score >= 0.1
RETURN a.wallet.address AS addr1,
       b.wallet.address AS addr2,
       r.relationship_score AS score,
       r.common_transactions AS common_tx,
       r.total_amount AS amount,
       r.avg_gas_used AS avg_gas,
       r.first_interaction AS first_interaction,
       r.last_interaction AS last_interaction,
       r.relationship_type AS rel_type,
       a.wallet.sui_balance AS addr1_balance,
       a.wallet.owned_objects_count AS addr1_objects,
       a.wallet.is_contract AS addr1_contract
LIMIT 50;

Fallback query: Inferring relationships from transaction edges (when there are no precomputed relationships)

When there are no precomputed related_to edges, relationships between wallets can be inferred by directly querying transaction edges. This is very useful for real-time or more granular transaction analysis.

USE sui_analysis;
MATCH (a:wallet)-[r:transaction]-(b:wallet)
RETURN a.wallet.address AS addr1,
       b.wallet.address AS addr2,
       r.amount AS amount,
       r.gas_used AS gas_used,
       r.success AS success,
       r.transaction_type AS tx_type,
       r.tx_timestamp AS tx_time,
       a.wallet.sui_balance AS addr1_balance,
       a.wallet.owned_objects_count AS addr1_objects,
       a.wallet.is_contract AS addr1_contract
LIMIT 200;

4.3 Address Details and Related Account Queries

The project allows users to query detailed information about a specific Sui wallet address and its directly related accounts, which is crucial for in-depth analysis of individual entity behavior patterns.

Querying target address basic information:

USE sui_analysis;
MATCH (target:wallet)
WHERE id(target) == "${searchAddress}"
RETURN target.wallet.address AS address,
       target.wallet.transaction_count AS tx_count,
       target.wallet.total_amount AS total_amount,
       target.wallet.first_seen AS first_seen,
       target.wallet.last_seen AS last_seen;

Querying related accounts of the target address:

USE sui_analysis;
MATCH (target:wallet)-[r:related_to]-(related:wallet)
WHERE id(target) == "${searchAddress}"
RETURN related.wallet.address AS address,
       r.relationship_score AS score,
       r.common_transactions AS common_tx,
       r.total_amount AS total_amount,
       r.first_interaction AS first_interaction,
       r.last_interaction AS last_interaction,
       r.relationship_type AS type
LIMIT 20;

Fallback: Analyzing relationships from transaction edges

When analyzing relationships of a specific address from raw transaction data, this query can be used.

USE sui_analysis;
MATCH (target:wallet)-[r:transaction]-(related:wallet)
WHERE id(target) == "${searchAddress}"
RETURN related.wallet.address AS address,
       related.wallet.transaction_count AS tx_count,
       related.wallet.total_amount AS amount,
       r.amount AS tx_amount,
       r.tx_timestamp AS tx_time
LIMIT 10;

4.4 Visualizing Network Connections

To provide an intuitive transaction network view, the project supports querying center nodes and their relationships, offering data support for frontend visualization.

Querying center node information:

USE sui_analysis;
MATCH (center:wallet)
WHERE id(center) == "${searchAddress}"
RETURN center.wallet.address AS center_address,
       center.wallet.transaction_count AS center_tx_count,
       center.wallet.total_amount AS center_amount;

Querying network relationships:

USE sui_analysis;
MATCH (center:wallet)-[r:transaction]-(connected:wallet)
WHERE id(center) == "${searchAddress}"
RETURN connected.wallet.address AS connected_address,
       connected.wallet.transaction_count AS connected_tx_count,
       connected.wallet.total_amount AS connected_amount,
       r.amount AS edge_amount,
       r.tx_timestamp AS tx_time
LIMIT 50;

V. Summary

The Sui-Nebula-Analyzer project successfully combines Sui blockchain data with the NebulaGraph graph database to build an efficient and flexible on-chain relationship analysis platform demo.

Although it's just a demo project, the demonstrated engineering architecture is effective and showcases NebulaGraph's powerful functionality and scalability in such scenarios. In a production environment, more graph algorithms like community detection and influence analysis could be integrated on this basis to achieve more precise relationship analysis.