graphprotocol/graph-node
Graph Node indexes data from blockchains such as Ethereum and serves it over GraphQL
π Critical Fix
- `EntityCache::load_related` returned wrong derived-collection membership on same-block parent reassignment. Membership was decided against intermediate cache layers rather than the entity's final state, so entities reassigned to a new parent within the same block could be left in the old collection, omitted from the new one, or wrongly retained after a cross-handler revert. See Upgrade Notes. ([#6548](https://github.com/graphprotocol/graph-node/pull/6548))
π₯ Breaking Changes
- Elasticsearch CLI flags removed. `--elasticsearch-url`, `--elasticsearch-user`, `--elasticsearch-password` (and the matching `ELASTICSEARCH_*` env vars) are gone. Configure Elasticsearch under the new `[log_store]` section in `graph-node.toml`. See Upgrade Notes. ([#6278](https://github.com/graphprotocol/graph-node/pull/6278))
β¨ What's New
- Query subgraph logs over GraphQL. A new `_logs` field exposes subgraph-emitted logs (mapping `log.*` calls, runtime, system) with filters for level, timestamp range, and text search, plus pagination and `orderDirection`. The old Elasticsearch sink for subgraph logs is replaced by a unified `LogStore` abstraction with three backends configured under a new `[log_store]` section in `graph-node.toml`: File (JSON Lines), Loki, or Elasticsearch. Omit `[log_store]` to disable; logs still go to stdout/stderr. ([#6278](https://github.com/graphprotocol/graph-node/pull/6278))
π¦ Improvements
- Revert entity versions during copy instead of in a separate post-copy pass, speeding up subgraph copy and graft. ([#6472](https://github.com/graphprotocol/graph-node/pull/6472))
π Bug Fixes
- Fixed `trace_filter` deserialization failing on providers that omit `result.output` (notably Sonic), which caused repeated trace ingestion retries. Fixed upstream in alloy 2.0.5 ([alloy#3931](https://github.com/alloy-rs/alloy/pull/3931)) and picked up via the bump in this release. ([#6576](https://github.com/graphprotocol/graph-node/pull/6576))
- Fixed runner panics on out-of-range `BigInt` values: `to_signed_u256` replaced by fallible `to_i256`, and `to_unsigned_u256` now errors on values `>= 2^256` instead of panicking. ([#6560](https://github.com/graphprotocol/graph-node/pull/6560))
- Fixed `graphman chain change-shard` failing with a duplicate-key error on revert to the original shard. Backups now use unique names and revert reuses the existing `<chain>-old`. ([#6199](https://github.com/graphprotocol/graph-node/pull/6199))
π¦ Upgrade Notes
- v0.44.0 changes `EntityCache::load_related` results for same-block parent-reassignment edge cases ([#6548](https://github.com/graphprotocol/graph-node/pull/6548)). Subgraphs that read derived fields within handlers may diverge in POI from v0.43.0 on blocks exhibiting this pattern; resync from before the affected blocks if affected. The v0.44.0 result is canonical.
- If you set `--elasticsearch-*` flags or `ELASTICSEARCH_*` env vars, migrate to a `[log_store]` section in `graph-node.toml` before upgrading. See `docs/log-store.md`. ([#6278](https://github.com/graphprotocol/graph-node/pull/6278))
π¦ Contributors
- Thanks to all contributors for this release: @erayack, @fordN, @incrypto32, @lutter
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.43.0...v0.44.0
β¨ What's New
- `skipDuplicates` for immutable entities. A new `skipDuplicates` parameter on the `@entity` directive (`@entity(immutable: true, skipDuplicates: true)`) silently skips duplicate inserts instead of failing the subgraph. This unblocks subgraph composition on Amp-powered subgraphs where SQL queries can produce the same entities across block ranges. ([#6458](https://github.com/graphprotocol/graph-node/pull/6458))
- Per-chain RPC settings via TOML config. Chain-specific tuning parameters (`json_rpc_timeout`, `request_retries`, `max_block_range_size`, `polling_interval`, `block_batch_size`, etc.) can now be set per chain in `config.toml` instead of relying on global environment variables. Fully backwards compatible β env vars remain the fallback. ([#6459](https://github.com/graphprotocol/graph-node/pull/6459))
- RPC provider failover for block ingestor. When the current RPC provider becomes unreachable during block ingestion polling, graph-node now automatically switches to a healthy alternative provider. ([#6430](https://github.com/graphprotocol/graph-node/pull/6430))
- Warn on startup when running a debug build, with `[DEBUG-BUILD]` prefix on all log lines. ([#6488](https://github.com/graphprotocol/graph-node/pull/6488))
π¦ Improvements
- Call cache eviction rewrite. Replaced the old per-contract iteration approach with ctid-based join deletes against the `call_meta` table, significantly faster on large `call_cache` tables. The old `--ttl-max-contracts` flag has been replaced by `--max-contracts`, which computes an effective TTL cutoff instead of limiting per-invocation iteration. Eviction now also returns stats on the number of contracts and entries removed. ([#6476](https://github.com/graphprotocol/graph-node/pull/6476)) ([#6477](https://github.com/graphprotocol/graph-node/pull/6477))
- Reduced unnecessary `eth_getBlockByHash` and `eth_getBlockByNumber` RPC calls by checking the block cache first in `block_pointer_from_number` and `fetch_full_block_with_rpc`. ([#6491](https://github.com/graphprotocol/graph-node/pull/6491), partially reverted by [#6537](https://github.com/graphprotocol/graph-node/pull/6537))
- Header-only `ChainStore` query methods (`ancestor_block_ptr`, `block_parent_ptr`) skip deserializing the full block `data` JSONB column when only hash/number/parent are needed. ([#6456](https://github.com/graphprotocol/graph-node/pull/6456))
- Batch checking for update attempts on immutable entities, reducing per-entity overhead.
- Node name is now used as the PostgreSQL `application_name` when `PGAPPNAME` is not set, making it easier to identify graph-node connections in `pg_stat_activity`.
π Bug Fixes
- Fixed dropped block trigger when `once` and `polling` filters match the same block β only one trigger type was firing. ([#6530](https://github.com/graphprotocol/graph-node/pull/6530))
- Fixed block stream ignoring configured `endBlock` in two cases: the block-skip optimization bypassing the `max_end_block` check, and `max_end_block` being set to `None` when multiple data sources share the same `endBlock`. ([#6474](https://github.com/graphprotocol/graph-node/pull/6474))
- Fixed GraphQL introspection not returning `isDeprecated: false` for `__InputValue`, which caused some client libraries to fail. ([#6475](https://github.com/graphprotocol/graph-node/pull/6475))
- Fixed IPC provider connections failing when configured with `ipc://` or `file://` URLs β the URL was passed directly to the transport instead of extracting the file path. ([#6443](https://github.com/graphprotocol/graph-node/pull/6443))
- Fixed `graphman config pools` not working due to hardcoded pool size override. ([#6444](https://github.com/graphprotocol/graph-node/pull/6444))
- Fixed unfail retry mechanism stopping after the first attempt when the deployment head was still behind the error block. ([#6529](https://github.com/graphprotocol/graph-node/pull/6529))
π¦ gnd (Graph Node Dev)
- `gnd indexer` command that delegates to `graph-indexer`, allowing indexer management (allocations, rules, cost models, status) directly through gnd. ([#6492](https://github.com/graphprotocol/graph-node/pull/6492))
- `gnd deploy` now prompts for `--version-label` in interactive mode and requires it in non-interactive mode. ([#6532](https://github.com/graphprotocol/graph-node/pull/6532))
- Test framework improvements: partial receipt.logs support, mock IPFS/Arweave clients for file data source testing. ([#6442](https://github.com/graphprotocol/graph-node/pull/6442))
π¦ Contributors
- Thanks to all contributors for this release: @aayushbaluni, @dimitrovmaksim, @incrypto32, @isum, @lutter, @suntzu
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.42.1...v0.43.0
π Bug Fixes
- Fixed multi-shard deployments becoming unresponsive due to `mirror_primary_tables` holding locks indefinitely, exhausting the connection pool. Added `statement_timeout` and `lock_timeout` to prevent stuck queries from cascading into a full lockup. (#6446)
- Fixed the `ingestor` setting in `[chains.*]` config sections having no effect β block ingestion ran regardless of its value. (#6447)
π₯ Breaking Changes
- Substreams support removed. Substreams have been unsupported on the network for some time. All substreams-related code has been removed, simplifying the codebase significantly. Substreams-based subgraphs will no longer work. ([#6261](https://github.com/graphprotocol/graph-node/pull/6261))
- Migrated from rust-web3 to alloy. The Ethereum RPC layer now uses the alloy crate instead of rust-web3. This should be transparent to most users, but chains that do not return EIP-2718 typed transaction fields may need the `no_eip2718` provider feature flag. ([#6063](https://github.com/graphprotocol/graph-node/pull/6063)) ([#6317](https://github.com/graphprotocol/graph-node/pull/6317))
- Note: This release includes two database migrations that run automatically on startup. If you need to roll back to v0.41.2, you will need to run the down migrations manually
β¨ What's New
- Amp-powered subgraphs (experimental). A new kind of subgraph powered by [Amp](https://github.com/edgeandnode/amp), the blockchain native database. Amp-powered subgraphs index data directly from Amp servers instead of processing blocks individually, reducing indexing time from days/weeks to minutes/hours. See `docs/amp-powered-subgraphs.md` for details. ([#6218](https://github.com/graphprotocol/graph-node/pull/6218)) ([#6293](https://github.com/graphprotocol/graph-node/pull/6293)) ([#6369](https://github.com/graphprotocol/graph-node/pull/6369))
- SQL query interface (experimental). A new experimental SQL query interface allows querying subgraph data using SQL syntax via GraphQL. Documentation in `docs/implementation/sql-interface.md`. ([#6172](https://github.com/graphprotocol/graph-node/pull/6172))
- Async store. All store and database interactions are now fully async using `diesel_async`, eliminating blocking database calls from tokio tasks. This improves throughput and reduces the risk of thread starvation under heavy load. ([#6194](https://github.com/graphprotocol/graph-node/pull/6194)) ([#6185](https://github.com/graphprotocol/graph-node/pull/6185))
- RPC request compression. Configurable compression for outgoing JSON-RPC requests to upstream providers, supporting gzip, brotli, and deflate. Configured via provider `features` in the config file. ([#6084](https://github.com/graphprotocol/graph-node/pull/6084))
- Query the current (partially filled) aggregation bucket with `current: include` in GraphQL requests. ([#6293](https://github.com/graphprotocol/graph-node/pull/6293))
- Extended `first` and `last` aggregation support to `String`, `Bytes`, and entity reference types. ([#6225](https://github.com/graphprotocol/graph-node/pull/6225))
- Automated account-like table optimization β an optional background job that detects and marks tables with high entity update ratios as account-like. Enabled via `GRAPH_STORE_ACCOUNT_LIKE_SCAN_INTERVAL_HOURS`, `GRAPH_STORE_ACCOUNT_LIKE_MIN_VERSIONS_COUNT`, and `GRAPH_STORE_ACCOUNT_LIKE_MAX_UNIQUE_RATIO`. ([#6209](https://github.com/graphprotocol/graph-node/pull/6209))
- Subgraph table sizes and row counts are now exposed in `SubgraphIndexingStatus` via the index-node API. ([#6201](https://github.com/graphprotocol/graph-node/pull/6201))
- + 3 more
π¦ Improvements
- Optimized WASM trigger instantiation by caching linker and compilation artifacts, reducing per-trigger overhead. ([#6364](https://github.com/graphprotocol/graph-node/pull/6364))
- Optimized log filter matching and trigger allocation, reducing CPU usage during block processing. ([#6419](https://github.com/graphprotocol/graph-node/pull/6419))
- Reduced unnecessary entity clones during block processing. ([#6362](https://github.com/graphprotocol/graph-node/pull/6362))
- Replaced `std::sync::RwLock` with `parking_lot::RwLock` across multiple components for better performance under contention.
- Deadlock prevention under heavy load, especially during index node startup β connection pool and store semaphore improvements. ([#6224](https://github.com/graphprotocol/graph-node/pull/6224))
- Raw subgraph manifests are now loaded from the store cache during startup, reducing IPFS dependency. ([#6223](https://github.com/graphprotocol/graph-node/pull/6223))
- Deployment hashes are now logged when removing a subgraph. ([#6405](https://github.com/graphprotocol/graph-node/pull/6405))
π Bug Fixes
- Fixed duplicate VID when multiple offchain triggers (file/IPFS data sources) fire in the same block, causing unique constraint violations. Also fixed in `ipfs.map()` callbacks. ([#6336](https://github.com/graphprotocol/graph-node/pull/6336)) ([#6416](https://github.com/graphprotocol/graph-node/pull/6416))
- Fixed incorrect `nocase` filter generation for non-String types (e.g., `Bytes`), which caused SQL errors like `operator does not exist: bytea ~~* bytea`. ([#6351](https://github.com/graphprotocol/graph-node/pull/6351))
- Fixed `can_copy_from` failures not fully rolling back deployment creation, leaving orphaned records. ([#6228](https://github.com/graphprotocol/graph-node/pull/6228))
- Fixed multi-column index validation bug where column order mismatch caused valid indexes to be silently dropped during subgraph copy/graft. ([#6341](https://github.com/graphprotocol/graph-node/pull/6341))
- Fixed grafting failure ("Unexpected null for non-null column") when source tables have too few rows for PostgreSQL to generate `histogram_bounds` statistics. ([#6275](https://github.com/graphprotocol/graph-node/pull/6275))
- Fixed panic when converting a negative number to `U256` in the runtime β now returns an error instead. ([#6219](https://github.com/graphprotocol/graph-node/pull/6219))
- Fixed `graphman copy` copying pruned `earliest_block_number` from source, which could leave the destination deployment in an invalid state. ([#6384](https://github.com/graphprotocol/graph-node/pull/6384))
- Fixed calls returning `0x` being incorrectly cached in the call cache, preventing stale empty results from persisting. ([#6187](https://github.com/graphprotocol/graph-node/pull/6187))
π¦ Graphman
- `graphman dump` and `graphman restore` (experimental). New commands for exporting and importing subgraph data in Parquet format, enabling backup and migration of deployment data across shards. Supports incremental dumps and progress reporting. See `docs/dump.md` for details. ([#6397](https://github.com/graphprotocol/graph-node/pull/6397))
- `graphman chain call-cache remove` now supports `--ttl-days` to remove stale call cache entries that haven't been accessed within a specified number of days. ([#6186](https://github.com/graphprotocol/graph-node/pull/6186))
π¦ gnd (Graph Node Dev)
- [experimental] Major expansion of `gnd` as a drop-in replacement for `graph-cli`: `init`, `add`, `codegen`, `build`, `publish`, `deploy`, `create`, `remove`, `auth`, `clean`, and `test` commands. ([#6282](https://github.com/graphprotocol/graph-node/pull/6282))
π¦ Contributors
- Thanks to all contributors for this release: @DaMandal0rian, @dimitrovmaksim, @fubhy, @hudsonhrh, @incrypto32, @isum, @lutter, @shiyasmohd
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.41.2...v0.42.0
π₯ Breaking Changes
- Substreams support removed. Substreams have been unsupported on the network for some time. All substreams-related code has been removed, simplifying the codebase significantly. Substreams-based subgraphs will no longer work. ([#6261](https://github.com/graphprotocol/graph-node/pull/6261))
- Migrated from rust-web3 to alloy. The Ethereum RPC layer now uses the alloy crate instead of rust-web3. This should be transparent to most users, but chains that do not return EIP-2718 typed transaction fields may need the `no_eip2718` provider feature flag. ([#6063](https://github.com/graphprotocol/graph-node/pull/6063)) ([#6317](https://github.com/graphprotocol/graph-node/pull/6317))
β¨ What's New
- Amp-powered subgraphs (experimental). A new kind of subgraph powered by [Amp](https://github.com/edgeandnode/amp), the blockchain native database. Amp-powered subgraphs index data directly from Amp servers instead of processing blocks individually, reducing indexing time from days/weeks to minutes/hours. See `docs/amp-powered-subgraphs.md` for details. ([#6218](https://github.com/graphprotocol/graph-node/pull/6218)) ([#6293](https://github.com/graphprotocol/graph-node/pull/6293)) ([#6369](https://github.com/graphprotocol/graph-node/pull/6369))
- SQL query interface (experimental). A new experimental SQL query interface allows querying subgraph data using SQL syntax via GraphQL. Documentation in `docs/implementation/sql-interface.md`. ([#6172](https://github.com/graphprotocol/graph-node/pull/6172))
- Async store. All store and database interactions are now fully async using `diesel_async`, eliminating blocking database calls from tokio tasks. This improves throughput and reduces the risk of thread starvation under heavy load. ([#6194](https://github.com/graphprotocol/graph-node/pull/6194)) ([#6185](https://github.com/graphprotocol/graph-node/pull/6185))
- RPC request compression. Configurable compression for outgoing JSON-RPC requests to upstream providers, supporting gzip, brotli, and deflate. Configured via provider `features` in the config file. ([#6084](https://github.com/graphprotocol/graph-node/pull/6084))
- Query the current (partially filled) aggregation bucket with `current: include` in GraphQL requests. ([#6293](https://github.com/graphprotocol/graph-node/pull/6293))
- Extended `first` and `last` aggregation support to `String`, `Bytes`, and entity reference types. ([#6225](https://github.com/graphprotocol/graph-node/pull/6225))
- Automated account-like table optimization β an optional background job that detects and marks tables with high entity update ratios as account-like. Enabled via `GRAPH_STORE_ACCOUNT_LIKE_SCAN_INTERVAL_HOURS`, `GRAPH_STORE_ACCOUNT_LIKE_MIN_VERSIONS_COUNT`, and `GRAPH_STORE_ACCOUNT_LIKE_MAX_UNIQUE_RATIO`. ([#6209](https://github.com/graphprotocol/graph-node/pull/6209))
- Subgraph table sizes and row counts are now exposed in `SubgraphIndexingStatus` via the index-node API. ([#6201](https://github.com/graphprotocol/graph-node/pull/6201))
- + 3 more
π¦ Improvements
- Optimized WASM trigger instantiation by caching linker and compilation artifacts, reducing per-trigger overhead. ([#6364](https://github.com/graphprotocol/graph-node/pull/6364))
- Optimized log filter matching and trigger allocation, reducing CPU usage during block processing. ([#6419](https://github.com/graphprotocol/graph-node/pull/6419))
- Reduced unnecessary entity clones during block processing. ([#6362](https://github.com/graphprotocol/graph-node/pull/6362))
- Replaced `std::sync::RwLock` with `parking_lot::RwLock` across multiple components for better performance under contention.
- Deadlock prevention under heavy load, especially during index node startup β connection pool and store semaphore improvements. ([#6224](https://github.com/graphprotocol/graph-node/pull/6224))
- Raw subgraph manifests are now loaded from the store cache during startup, reducing IPFS dependency. ([#6223](https://github.com/graphprotocol/graph-node/pull/6223))
- Deployment hashes are now logged when removing a subgraph. ([#6405](https://github.com/graphprotocol/graph-node/pull/6405))
π Bug Fixes
- Fixed duplicate VID when multiple offchain triggers (file/IPFS data sources) fire in the same block, causing unique constraint violations. Also fixed in `ipfs.map()` callbacks. ([#6336](https://github.com/graphprotocol/graph-node/pull/6336)) ([#6416](https://github.com/graphprotocol/graph-node/pull/6416))
- Fixed incorrect `nocase` filter generation for non-String types (e.g., `Bytes`), which caused SQL errors like `operator does not exist: bytea ~~* bytea`. ([#6351](https://github.com/graphprotocol/graph-node/pull/6351))
- Fixed `can_copy_from` failures not fully rolling back deployment creation, leaving orphaned records. ([#6228](https://github.com/graphprotocol/graph-node/pull/6228))
- Fixed multi-column index validation bug where column order mismatch caused valid indexes to be silently dropped during subgraph copy/graft. ([#6341](https://github.com/graphprotocol/graph-node/pull/6341))
- Fixed grafting failure ("Unexpected null for non-null column") when source tables have too few rows for PostgreSQL to generate `histogram_bounds` statistics. ([#6275](https://github.com/graphprotocol/graph-node/pull/6275))
- Fixed panic when converting a negative number to `U256` in the runtime β now returns an error instead. ([#6219](https://github.com/graphprotocol/graph-node/pull/6219))
- Fixed `graphman copy` copying pruned `earliest_block_number` from source, which could leave the destination deployment in an invalid state. ([#6384](https://github.com/graphprotocol/graph-node/pull/6384))
- Fixed calls returning `0x` being incorrectly cached in the call cache, preventing stale empty results from persisting. ([#6187](https://github.com/graphprotocol/graph-node/pull/6187))
π¦ Graphman
- `graphman dump` and `graphman restore` (experimental). New commands for exporting and importing subgraph data in Parquet format, enabling backup and migration of deployment data across shards. Supports incremental dumps and progress reporting. See `docs/dump.md` for details. ([#6397](https://github.com/graphprotocol/graph-node/pull/6397))
- `graphman chain call-cache remove` now supports `--ttl-days` to remove stale call cache entries that haven't been accessed within a specified number of days. ([#6186](https://github.com/graphprotocol/graph-node/pull/6186))
π¦ gnd (Graph Node Dev)
- [experimental] Major expansion of `gnd` as a drop-in replacement for `graph-cli`: `init`, `add`, `codegen`, `build`, `publish`, `deploy`, `create`, `remove`, `auth`, `clean`, and `test` commands. ([#6282](https://github.com/graphprotocol/graph-node/pull/6282))
π¦ Contributors
- Thanks to all contributors for this release: @DaMandal0rian, @dimitrovmaksim, @fubhy, @hudsonhrh, @incrypto32, @isum, @lutter, @shiyasmohd
π¦ v0.41.2
- ```
- $ docker pull graphprotocol/graph-node:v0.41.2
- ```
π Bug Fixes
- Fix entity corruption after rewinding pruned subgraphs
- Fix rewind/truncate targeting the wrong deployment
- Fix batch write memoization bug
- Expand deterministic RPC error handling for Reth and Erigon
- Full changelog: https://github.com/graphprotocol/graph-node/compare/v0.41.1...v0.41.2
π¦ v0.41.1
- ```
- $ docker pull graphprotocol/graph-node:v0.41.1
- ```
π Bug Fix
- Indexing Status Endpoint
π¦ v0.41.0
- ```
- $ docker pull graphprotocol/graph-node:v0.41.0
- ```
π Bug Fixes
- Fix non-deterministic handling of transaction receipts in event handlers; receipts are only processed when explicitly declared (PR #6200)
β¨ New Features
- Added support for struct field access in declarative calls - enables accessing nested struct fields by field name using dot notation in manifest call declarations(#6099)
- New standalone `gnd` (Graph Node Dev) crate - a graph-node binary optimized for local development with minimal setup required (#6167)
- Extended IPFS usage metrics and logging capabilities (#6058)
- Extended support for additional IPFS content path formats (#6058)
π¦ Improvements
- Deferred IPFS manifest fetching when starting subgraphs - fixes issue where slow IPFS responses could block assignment event processing, preventing new subgraphs from being assigned (#6170)
- Added Content Identifier (CID) to IPFS retry logs for better debugging and tracking of IPFS operations (#6144)
- Runtime now uses interior mutability in `AscHeapCtx` for improved performance (#6053)
- Fixed handling of empty arrays in indexing status queries - prevents errors when querying deployments with empty array filters (#6143)
- Fixed case-insensitive array inputs in queries - array filter values are now properly case-normalized (#6075)
- Fixed panic when handling empty lists in `list_values` - resolves crashes with empty array filters (#6100)
- Enhanced logging for subgraph assignment processing with more detailed information (#6169)
- Added logging for `MAX_BLOCKING_THREADS` configuration setting (#6161)
- + 1 more
π¦ v0.41.0-rc.0
- ```
- $ docker pull graphprotocol/graph-node:v0.41.0-rc.0
- ```
β¨ New Features
- Added support for struct field access in declarative calls - enables accessing nested struct fields by field name using dot notation in manifest call declarations(#6099)
- New standalone `gnd` (Graph Node Dev) crate - a graph-node binary optimized for local development with minimal setup required (#6167)
- Extended IPFS usage metrics and logging capabilities (#6058)
- Extended support for additional IPFS content path formats (#6058)
π¦ Improvements
- Deferred IPFS manifest fetching when starting subgraphs - fixes issue where slow IPFS responses could block assignment event processing, preventing new subgraphs from being assigned (#6170)
- Added Content Identifier (CID) to IPFS retry logs for better debugging and tracking of IPFS operations (#6144)
- Runtime now uses interior mutability in `AscHeapCtx` for improved performance (#6053)
- Fixed handling of empty arrays in indexing status queries - prevents errors when querying deployments with empty array filters (#6143)
- Fixed case-insensitive array inputs in queries - array filter values are now properly case-normalized (#6075)
- Fixed panic when handling empty lists in `list_values` - resolves crashes with empty array filters (#6100)
- Enhanced logging for subgraph assignment processing with more detailed information (#6169)
- Added logging for `MAX_BLOCKING_THREADS` configuration setting (#6161)
- + 1 more
π¦ Improvements
- Enhanced IPFS logging to include Content Identifiers (CIDs) in operation names for better debugging
- Full changelog: https://github.com/graphprotocol/graph-node/compare/v0.40.0...v0.40.1
π¦ Contributors
- Thanks to @fubhy and all other contributors for their work on this release.
π Critical Bugfix
- GraphQL Empty Array Panic Fix
β¨ New Features
- IPFS File System Caching
- IPFS files can now be cached to disk using the `GRAPH_IPFS_CACHE_LOCATION` environment variable - reduces IPFS requests after restarts and improves performance. Cache directory must be writable by graph-node and in a trusted location (#6031)
π¦ Improvements
- Performance
- Speed up appending changes to batch operations (#6025)
- Improved backoff strategy for offchain data sources - files not found are subject to exponential backoff, with configurable maximum backoff via `GRAPH_FDS_MAX_BACKOFF` (default: 600 seconds) (#6043)
- GraphQL
- Better error messages for OR operator usage with column filters (#6078)
- graphman
- Remove create flag from `graphman deploy` command (#6077)
- `graphman reassign` now shows clearer success messages (#6074)
- + 2 more
π Bug Fixes
- Fixed subgraph composition sync failures when entities have different ID types - resolves "UNION types bytea and text cannot be matched" error (#6080)
- Fixed database down migration issue (#6065)
- Fixed pruning status reporting accuracy - completion percentage now correctly reflects progress when copying non-final entities (#6062)
- Full changelog: <https://github.com/graphprotocol/graph-node/compare/v0.39.1...v0.40.0>
π¦ Contributors
- Thanks to @lutter, @mangas, @fubhy, @shiyasmohd, and all other contributors for their work on this release.
β¨ New Features
- IPFS File System Caching
- IPFS files can now be cached to disk using the `GRAPH_IPFS_CACHE_LOCATION` environment variable - reduces IPFS requests after restarts and improves performance. Cache directory must be writable by graph-node and in a trusted location (#6031)
π¦ Improvements
- Performance
- Speed up appending changes to batch operations (#6025)
- Improved backoff strategy for offchain data sources - files not found are subject to exponential backoff, with configurable maximum backoff via `GRAPH_FDS_MAX_BACKOFF` (default: 600 seconds) (#6043)
- GraphQL
- Better error messages for OR operator usage with column filters (#6078)
- graphman
- Remove create flag from `graphman deploy` command (#6077)
- `graphman reassign` now shows clearer success messages (#6074)
- + 2 more
π Bug Fixes
- Fixed subgraph composition sync failures when entities have different ID types - resolves "UNION types bytea and text cannot be matched" error (#6080)
- Fixed database down migration issue (#6065)
- Fixed pruning status reporting accuracy - completion percentage now correctly reflects progress when copying non-final entities (#6062)
- Full changelog: <https://github.com/graphprotocol/graph-node/compare/v0.39.1...v0.40.0-rc.0>
π¦ Contributors
- Thanks to @lutter, @mangas, @fubhy, @shiyasmohd, and all other contributors for their work on this release.
π Critical Fix
- Reverted transaction log index behavior: Fixed a regression introduced in v0.38.0 where `event.transactionLogIndex` was inadvertently changed to return the actual transaction log index instead of the log index ([#6042](https://github.com/graphprotocol/graph-node/pull/6042))
- ---
π¦ Upgrade Notes
- Indexers running v0.38.0 who have synced subgraphs that use `event.transactionLogIndex` should:
- 1. Upgrade to v0.39.1
- 2. Rewind affected subgraphs to before they were synced with v0.38.0
- 3. Resync to ensure POI convergence with the network
- ---
π¦ Contributors
- Thanks to @lutter for the quick fix and the indexer community for identifying and helping diagnose this issue.
- ---
- Full changelog: <https://github.com/graphprotocol/graph-node/compare/v0.39.0...v0.39.1>
π₯ Breaking Changes
- Major schema migration: The `subgraphs.subgraph_deployment` table has been split into two tables: ([#6003](https://github.com/graphprotocol/graph-node/pull/6003))
- `subgraphs.head` - stores frequently changing metadata (block hash, block number, entity count, firehose cursor)
- `subgraphs.deployment` - stores less frequently changing data
- Arweave blockchain support removed - Arweave subgraphs are no longer supported (Arweave file data sources remain unaffected) ([#5951](https://github.com/graphprotocol/graph-node/pull/5951))
- `graphman drop` command removed - To remove a deployment, run `graphman remove <name>` for each name the deployment has (see `graphman info`), then run `graphman unused record && graphman unused remove` ([#5974](https://github.com/graphprotocol/graph-node/pull/5974))
- Failed subgraphs are now paused instead of unassigned - This prevents unwanted side effects like canceling active copy operations ([#5971](https://github.com/graphprotocol/graph-node/pull/5971))
- Various store errors are now classified as deterministic - May affect error handling in your monitoring ([#5943](https://github.com/graphprotocol/graph-node/pull/5943))
- ---
β¨ New Features
- Pruning status tracking: New database tables track pruning progress per deployment ([#5949](https://github.com/graphprotocol/graph-node/pull/5949))
- Pruning timeout protection: Long-running prune operations can be killed and retried if they exceed `GRAPH_STORE_BATCH_TIMEOUT` ([#6002](https://github.com/graphprotocol/graph-node/pull/6002))
- Better pruning estimates: Uses PostgreSQL statistics for more accurate predictions (requires PostgreSQL 17+ for full functionality) ([#6012](https://github.com/graphprotocol/graph-node/pull/6012))
- New graphman commands: ([#5949](https://github.com/graphprotocol/graph-node/pull/5949))
- `graphman prune status` - View status of prune operations
- `graphman prune run` - Manually trigger pruning
- `graphman prune set` - Configure pruning parameters
- `graphman chain ingest <chain_name> <block_number>` - Manually ingest specific blocks into the block cache ([#5945](https://github.com/graphprotocol/graph-node/pull/5945))
- + 5 more
π¦ Improvements
- Store errors now properly classified as deterministic vs non-deterministic ([#5943](https://github.com/graphprotocol/graph-node/pull/5943))
- Pruning errors no longer cause subgraph failure - they're logged and retried later ([#5972](https://github.com/graphprotocol/graph-node/pull/5972))
- Better error messages for:
- Source subgraph manifest resolution ([#5950](https://github.com/graphprotocol/graph-node/pull/5950))
- Store write failures
- Unique constraint violations ([#5943](https://github.com/graphprotocol/graph-node/pull/5943))
- Optimized GraphQL result caching ([#6006](https://github.com/graphprotocol/graph-node/pull/6006))
- Reduced code duplication in subgraph processing ([#5952](https://github.com/graphprotocol/graph-node/pull/5952))
- + 6 more
π Bug Fixes
- Fixed VID sequence naming to comply with PostgreSQL 63-character limit ([a322a634](https://github.com/graphprotocol/graph-node/commit/a322a63455421a94429d3ca9f7543bfca6b2086d))
- Fixed pruning of tables with VID sequences using CASCADE drops ([#5968](https://github.com/graphprotocol/graph-node/pull/5968))
- PostgreSQL unique constraint violations now lead to deterministic subgraph failures ([#5943](https://github.com/graphprotocol/graph-node/pull/5943))
- Fixed various edge cases in error classification ([#5943](https://github.com/graphprotocol/graph-node/pull/5943))
- Fixed numerical precision issues with large VID values ([#5970](https://github.com/graphprotocol/graph-node/pull/5970))
- ---
π Configuration Changes
- `GRAPH_IPFS_MAX_ATTEMPTS` - Maximum IPFS retry attempts (default: 100,000)
- `GRAPH_IPFS_REQUEST_TIMEOUT` - IPFS request timeout in seconds
- `GRAPH_STORE_BATCH_TIMEOUT` - Timeout for batch operations like pruning
- `GRAPH_STORE_HISTORY_KEEP_STATUS` - Number of pruning status records to keep (default: 5)
- `GRAPH_STORE_PRUNE_DISABLE_RANGE_BOUND_ESTIMATION` - Disable new pruning estimation method (temporary compatibility flag)
- `GRAPH_STORE_ERRORS_ARE_NON_DETERMINISTIC` - Revert to treating all errors as non-deterministic (temporary compatibility flag)
- ---
π¦ Contributors
- Thanks to @lutter, @incrypto32, @zorancv, @shiyasmohd, @encalypto, @dwerner, @rotarur, and all other contributors for their work on this release.
- ---
- Full changelog: <https://github.com/graphprotocol/graph-node/compare/v0.38.0...v0.39.0>
π₯ Breaking changes
- Removal of GraphQL subscriptions β the WebSocket server and every `Subscription`βrelated code path were deleted. Applications that need live updates must switch to polling or an external event system. ([#5836](https://github.com/graphprotocol/graph-node/pull/5836))
- Removal of Cosmos support β all Cosmos chain and runtime code was removed. ([#5833](https://github.com/graphprotocol/graph-node/pull/5833))
- `GRAPH_STORE_LAST_ROLLUP_FROM_POI` deleted β this setting is no longer needed and will be ignored. ([#5936](https://github.com/graphprotocol/graph-node/pull/5936))
- No dependency on `pg_stat_statements` β GraphΒ Node no longer executes `pg_stat_statements_reset()` after running database migrations and therefore is agnostic to whether that extension is installed or not ([#5926](https://github.com/graphprotocol/graph-node/pull/5926))
- ---
β¨ New features & improvements
- 1. Faster grafting and copying
- A single graft/copy operation can now copy multiple tables in parallel. The number of parallel operations per graft/copy can be configured by setting `GRAPH_STORE_BATCH_WORKERS`, which defaults to 1. The total number of parallel operations is also limited by the `fdw_pool_size` which defaults to 5 and can be set in `graph-node.toml`
- A large number of parallel grafts/copies might get blocked by the size of tokio's blocking thread pool. To avoid that block, the size of that pool can now be configured with `GRAPH_MAX_BLOCKING_THREADS`. It defaults to 512 (tokio's default) but setting this to a large number like 2048 should be safe. ([#5948](https://github.com/graphprotocol/graph-node/pull/5948))
- 3. Better control of graft/copy batches
- To avoid table bloat, especially in `subgraphs.subgraph_deployment`, graft/copy splits the work up into smaller batches which should take `GRAPH_STORE_BATCH_TARGET_DURATION` each. Sometimes, the estimation for how long a batch will take goes horribly wrong. To guard against that, the new setting `GRAPH_STORE_BATCH_TIMEOUT` sets a timeout, which is unlimited by default. When set, batches that take longer than this are aborted and restarted with a much smaller size. ([3c183731](https://github.com/graphprotocol/graph-node/commit/3c18373109d16dadab2d9c52fdd5f81cf3de6dbe))
- The number of rows that are fetched from the source shard for cross-shard grafts/copies in a single operation can be controlled through `GRAPH_STORE_FDW_FETCH_SIZE`. Its default has been lowered from 10Β 000Β toΒ 1Β 000, as larger sizes have shown to not be effective and actually cause the graft/copy to slow down in some cases ([#5924](https://github.com/graphprotocol/graph-node/pull/5924))
- Deployments being copied are excluded from pruning. ([#5893](https://github.com/graphprotocol/graph-node/pull/5893))
- Failed deployments can now be copied. ([#5893](https://github.com/graphprotocol/graph-node/pull/5893))
- + 15 more
π Fixes (selected)
- Aggregate indexes were sometimes created twice when postponed. ([4be64c16](https://github.com/graphprotocol/graph-node/commit/4be64c16f575c9da424ac730e26a497f829683ee))
- Duplicate `remove` operations in a write batch no longer cause failures. ([17360f56](https://github.com/graphprotocol/graph-node/commit/17360f56c7657e49d2b5da2fafa5d8d633d556f7))
- Incorrect hashing when grafting from subgraphs with `specVersion`Β <Β 0.0.6 fixed. ([#5917](https://github.com/graphprotocol/graph-node/pull/5917))
- Firehose TLS configuration corrected. ([36ad6a24](https://github.com/graphprotocol/graph-node/commit/36ad6a24a2456fa5504f14b20dd59c0cffae2b40))
- Numerous small fixes in estimation of graft/copy batch size, namespace mapping, copy status display, and error messages.
- ---
π¦ Contributors
- @lutter, @zorancv, @incrypto32, @filipeazevedo, @encalypto, @shiyasmohd, and many others β thank you for your work on this release.
- ---
- Full changelog: <https://github.com/graphprotocol/graph-node/compare/v0.37.0...v0.38.0>
β¨ What's new
- Composable Subgraphs: A feature that enables subgraphs to use other subgraphs as data sources. This introduces modularity in subgraph development by allowing developers to create source subgraphs that can be referenced by dependent subgraphs. The feature supports up to 5 subgraph datasources.
- Note for Indexers: To deploy a dependent subgraph, the source subgraph(s) must already be deployed and available on your infrastructure.
- YAML Parsing in Subgraph Mappings: Added YAML parsing support to subgraph mappings through new host functions, enabling subgraphs to parse YAML documents (including subgraph manifests) directly in their mappings
π¦ Improvements
- Added new deployment metrics:
- `deployment_status`: Tracks deployment lifecycle (starting, running, stopped, failed)
- `deployment_synced`: Indicates if deployment has reached chain head
- Optimized copy, graft, and prune operations for unevenly distributed rows
- Improved error handling for substreams-powered subgraphs
- Enhanced error handling when restarting paused subgraphs
- Improved Firehose block request handling with retries and block cache
- <!-- - Various VID-related fixes and improvements -->
- + 1 more
π¦ Graphman
- Added unassign/reassign commands to GraphQL API
π₯ Breaking changes
- The `deployment_failed` metric has been removed in favor of the new `deployment_status` metric
π¦ Contributors
- Thanks to @zorancv, @shiyasmohd, @isum, @DaMandal0rian, @incrypto32, @King-witcher, and @lutter for contributing to this release.
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.36.1...v0.37.0
π¦ *Note for indexers upgrading from a manually-downgraded version*
- >
- > ```
- > store/postgres/migrations/2024-06-11-084227_track-more-features-in-subgraph-features/down.sql
- > store/postgres/migrations/2024-07-22-140930_track_synced_date/down.sql
- > store/postgres/migrations/2024-08-14-205601_store_synced_at_block/down.sql
- > store/postgres/migrations/2024-10-01-100427_create_graphman_command_executions_table/down.sql
- > ```
- >
- + 15 more
π¦ Improvements
- This release addresses several issues found in `v0.36.0`, notably:
- Remove confusing IPFS warning messages [(#5723)]
- Better concurrent IPFS client selection [(#5723)]
- Bug with handling of enum arrays that can cause subgraphs to fail [(#5372)]
- Additionally, this release has been tested for an extended period on the Upgrade Indexer, ensuring
- issues in `v0.36.0` related to IPFS, block ingestion, and migrations work as intended.
- Add subcommand description to `graphman deploy` [(#5744)]
- Display log message argument when calling `log.critical` [(#5744)]
- + 6 more
π¦ Note on Firehose Extended Block Details
- To disable checks for one or more chains, simply specify their names
- in `GRAPH_NODE_FIREHOSE_DISABLE_EXTENDED_BLOCKS_FOR_CHAINS` as a comma separated list of chain names. Graph Node defaults to an empty list, which means that this feature is enabled for all chains.
β¨ What's new
- Add support for substreams using 'index modules', 'block filters', 'store:sum_set'. [(#5463)](https://github.com/graphprotocol/graph-node/pull/5463)
- Implement new IPFS client [(#5600)](https://github.com/graphprotocol/graph-node/pull/5600)
- Add `timestamp` support to substreams. [(#5641)](https://github.com/graphprotocol/graph-node/pull/5641)
- Add graph-indexed header to query responses. [(#5710)](https://github.com/graphprotocol/graph-node/pull/5710)
- Use the new Firehose info endpoint. [(#5672)](https://github.com/graphprotocol/graph-node/pull/5672)
- Store `synced_at_block_number` when a deployment syncs. [(#5610)](https://github.com/graphprotocol/graph-node/pull/5610)
- Create nightly docker builds from master branch. [(#5400)](https://github.com/graphprotocol/graph-node/pull/5400)
- Make sure `transact_block_operations` does not go backwards. [(#5419)](https://github.com/graphprotocol/graph-node/pull/5419)
- + 30 more
π Bug fixes
- Add safety check when rewinding. [(#5423)](https://github.com/graphprotocol/graph-node/pull/5423)
- Fix rewind for deployments with multiple names. [(#5502)](https://github.com/graphprotocol/graph-node/pull/5502)
- Improve `graphman copy` performance [(#5425)](https://github.com/graphprotocol/graph-node/pull/5425)
- Fix retrieving chain info with graphman for some edge cases. [(#5516)](https://github.com/graphprotocol/graph-node/pull/5516)
- Improve `graphman restart` to handle multiple subgraph names for a deployment. [(#5674)](https://github.com/graphprotocol/graph-node/pull/5674)
- Improve adapter startup. [(#5503)](https://github.com/graphprotocol/graph-node/pull/5503)
- Detect Nethermind eth_call reverts. [(#5533)](https://github.com/graphprotocol/graph-node/pull/5533)
- Fix genesis block fetching for substreams. [(#5548)](https://github.com/graphprotocol/graph-node/pull/5548)
- + 17 more
π¦ Graphman
- Add command to update genesis block for a chain and to check genesis information against all providers. [(#5517)](https://github.com/graphprotocol/graph-node/pull/5517)
- Create GraphQL API to execute commands [(#5554)](https://github.com/graphprotocol/graph-node/pull/5554)
- Add graphman create/remove commands to GraphQL API. [(#5685)](https://github.com/graphprotocol/graph-node/pull/5685)
π¦ Contributors
- Thanks to all contributors for this release: @dwerner, @encalypto, @incrypto32, @isum, @leoyvens, @lutter, @mangas, @sduchesneau, @Shiyasmohd, @shuaibbapputty, @YaroShkvorets, @ziyadonji, @zorancv
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.35.1...v0.36.0
β¨ New subgraph features
- Declarative Eth calls - support executing Eth calls ahead of time and in parallel. Using this feature, the time for all calls goes from the sum of the time the calls take to the max. [#5264](https://github.com/graphprotocol/graph-node/pull/5264)
- Support for differentiating EOA from smart contract accounts - ethereum.getCode can be used in mappings to check whether smart contract code is present at a given address. [#5272](https://github.com/graphprotocol/graph-node/pull/5272)
- Support additional event filters - allow filtering on Indexed arguments. [#5254](https://github.com/graphprotocol/graph-node/pull/5254)
- Aggregations now use `Timestamp` type for the `timestamp` attribute. [#5321](https://github.com/graphprotocol/graph-node/pull/5321)
π¦ Optimizations
- Fetch block receipts in bulk - use eth_getBlockReceipts on supported providers to fetch receipts in bulk. [#5341](https://github.com/graphprotocol/graph-node/pull/5341)
- Improve indices created to support aggregations. [#5286](https://github.com/graphprotocol/graph-node/pull/5286)
- Use 'select by specific attributes' by default. [#5311](https://github.com/graphprotocol/graph-node/pull/5311)
- Select only the needed specific attributes for query_window_one_entity. [#5357](https://github.com/graphprotocol/graph-node/pull/5357)
- Check interface uniqueness in bulk. [#5293](https://github.com/graphprotocol/graph-node/pull/5293)
- Stop using the IPFS `stat` API in favor of the more efficient `cat` API. [#5284](https://github.com/graphprotocol/graph-node/pull/5284)
- Derive macros for `CheapClone` and `CacheWeight`. [#5326](https://github.com/graphprotocol/graph-node/pull/5326)
- Periodically remove entries from the `LayoutCache`. [#5324](https://github.com/graphprotocol/graph-node/pull/5324)
- + 2 more
π¦ Graphman
- Refactor manual index creation & add ddl tests. [#5292](https://github.com/graphprotocol/graph-node/pull/5292)
- Refactor `graphman rewind` to use pause and resume logic. [#5213](https://github.com/graphprotocol/graph-node/pull/5213)
π¦ Firehose & Substreams
- Add deployment id to Firehose connection headers. [#5319](https://github.com/graphprotocol/graph-node/pull/5319)
- Improve the Ethereum Firehose codec. [#5314](https://github.com/graphprotocol/graph-node/pull/5314)
- Fix static filters restart behaviour when using threshold. [#5328](https://github.com/graphprotocol/graph-node/pull/5328)
- Try to be more consistent in Firehose parsing to address. [#5360](https://github.com/graphprotocol/graph-node/pull/5360)
- Include 'provider' in various firehose log messages. [#5384](https://github.com/graphprotocol/graph-node/pull/5384)
- Do not send all Firehose headers when its init triggers only. [#5373](https://github.com/graphprotocol/graph-node/pull/5373)
π¦ Robustness
- Add a test for number_gte block constraints. [#5383](https://github.com/graphprotocol/graph-node/pull/5383)
- Add zkSync fingerprint to the `TOO_MANY_LOGS_FINGERPRINTS`. [#5307](https://github.com/graphprotocol/graph-node/pull/5307)
- Add exceptional case for transaction less event in zksync-era. [#5338](https://github.com/graphprotocol/graph-node/pull/5338)
- Improve tests by moving subgraph param to TestContext. [#5342](https://github.com/graphprotocol/graph-node/pull/5342)
π¦ Notable dependency upgrades
- Upgrade clap. [#5174](https://github.com/graphprotocol/graph-node/pull/5174)
- Update syn to 2.0. [#5339](https://github.com/graphprotocol/graph-node/pull/5339)
- Update tokio-tungstenite. [#5344](https://github.com/graphprotocol/graph-node/pull/5344)
π Bug fixes
- Avoid 'too many bind params' error in FindDerivedEntityQuery. [#5318](https://github.com/graphprotocol/graph-node/pull/5318)
- Fix check for deterministic error. [#5332](https://github.com/graphprotocol/graph-node/pull/5332)
- Fix incorrect comparison of transaction hash. [#5374](https://github.com/graphprotocol/graph-node/pull/5374)
- Fix clap. [#5389](https://github.com/graphprotocol/graph-node/pull/5389)
π¦ Contributors
- @itsjerryokolo made their first contribution in [#5213](https://github.com/graphprotocol/graph-node/pull/5213)! Thanks for the contribution @itsjerryokolo!π
- Thanks to all contributors to this release: @incrypto32, @lutter, @leoyvens, @mangas, @itsjerryokolo, @YaroShkvorets, @fordN, and of course the trusty @dependabot.
- For readability not all commits and changes were noted here. To see a complete list head to the full changelog at https://github.com/graphprotocol/graph-node/compare/v0.35.0...v0.35.1
β¨ What's new
- Aggregations - Declarative aggregations defined in the subgraph schema allow the developer to aggregate values on specific intervals using flexible aggregation functions. [(#5082)](https://github.com/graphprotocol/graph-node/pull/5082) [(#5184)](https://github.com/graphprotocol/graph-node/pull/5184) [(#5209)](https://github.com/graphprotocol/graph-node/pull/5209) [(#5242)](https://github.com/graphprotocol/graph-node/pull/5242) [(#5208)](https://github.com/graphprotocol/graph-node/pull/5208)
- Add pause and resume to admin JSON-RPC API - Adds support for explicit pausing and resuming of subgraph deployments with a field tracking the paused state in `indexerStatuses`. [(#5190)](https://github.com/graphprotocol/graph-node/pull/5190)
- Support eth_getBalance calls in subgraph mappings - Enables fetching the Eth balance of an address from the mappings using `ethereum.getBalance(address)`. [(#5202)](https://github.com/graphprotocol/graph-node/pull/5202)
- Add parentHash to _meta query - Particularly useful when polling for data each block to verify the sequence of blocks. [(#5232)](https://github.com/graphprotocol/graph-node/pull/5232)
- Parallel execution of all top-level queries in a single query body [(#5273)](https://github.com/graphprotocol/graph-node/pull/5273)
- The ElasticSearch index to which `graph-node` logs can now be configured with the `GRAPH_ELASTIC_SEARCH_INDEX` environment variable which defaults to `subgraph`. [(#5210)](https://github.com/graphprotocol/graph-node/pull/5210)
- Some small prefetch simplifications. [(#5132)](https://github.com/graphprotocol/graph-node/pull/5132)
- Migration changing the type of health column to text. [(#5077)](https://github.com/graphprotocol/graph-node/pull/5077)
- + 15 more
π¦ Graphman
- Add ability to list removed unused deployment by id. [(#5152)](https://github.com/graphprotocol/graph-node/pull/5152)
- Add command to change block cache shard. [(#5169)](https://github.com/graphprotocol/graph-node/pull/5169)
π¦ Firehose and Substreams
- Add key-based authentication for Firehose/Substreams providers. [(#5259)](https://github.com/graphprotocol/graph-node/pull/5259)
- Increase blockstream buffer size for substreams. [(#5182)](https://github.com/graphprotocol/graph-node/pull/5182)
- Improve substreams error handling. [(#5160)](https://github.com/graphprotocol/graph-node/pull/5160)
- Reset substreams/firehose block ingestor backoff. [(#5047)](https://github.com/graphprotocol/graph-node/pull/5047)
π Bug Fixes
- Fix graphiql issue when querying subgraph names with multiple path segments. [(#5136)](https://github.com/graphprotocol/graph-node/pull/5136)
- Fix change_health_column migration for sharded setup. [(#5183)](https://github.com/graphprotocol/graph-node/pull/5183)
- Fix conversion of BlockTime for NEAR. [(#5206)](https://github.com/graphprotocol/graph-node/pull/5206)
- Call revert_state_to to last good block instead of current block. [(#5195)](https://github.com/graphprotocol/graph-node/pull/5195)
- Fix Action::block_finished. [(#5218)](https://github.com/graphprotocol/graph-node/pull/5218)
- Fix runtime timeouts. [(#5236)](https://github.com/graphprotocol/graph-node/pull/5236)
- Remove panic from rewind and truncate. [(#5233)](https://github.com/graphprotocol/graph-node/pull/5233)
- Fix version stats for huge number of versions. [(#5261)](https://github.com/graphprotocol/graph-node/pull/5261)
- + 1 more
π¦ Major dependency upgrades
- Update to diesel v2. [(#5002)](https://github.com/graphprotocol/graph-node/pull/5002)
- Bump Rust version. [(#4985)](https://github.com/graphprotocol/graph-node/pull/4985)
π¦ Contributors
- Thank you to all the contributors! `@incrypto32`, `@mangas`, `@lutter`, `@leoyvens`, `@zorancv`, `@YaroShkvorets`, `@seem-less`
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.34.1...v0.35.0
π Changes
- Fixes an issue that caused an increase in data size of `/metrics` endpoint of graph-node. [(#5161)](https://github.com/graphprotocol/graph-node/issues/5161). Indexers are advised to skip `v0.34.0` and use `v0.34.1`
- Fixes an issue that caused subgraphs with file data sources to skip non-deterministic errors that occurred in a file data source mapping handler.
β¨ What's New
- Substreams as Source of Triggers for Subgraphs - This update significantly enhances subgraph functionality by enabling substreams to act as a source of triggers for running subgraph mappings. Developers can now directly run subgraph mappings on the data output from substreams, facilitating a more integrated and efficient workflow.[(#4887)](https://github.com/graphprotocol/graph-node/pull/4887) [(#4916)](https://github.com/graphprotocol/graph-node/pull/4916)
- `indexerHints` in Manifest for Automated Pruning - This update introduces the ability for subgraph authors to specify `indexerHints` with a field `prune` in their manifest, indicating the desired extent of historical block data retention. This feature enables graph-node to automatically prune subgraphs when the stored history exceeds the specified limit, significantly improving query performance. This automated process eliminates the need for manual action by indexers for each subgraph. Indexers can also override user-set historyBlocks with the environment variable `GRAPH_HISTORY_BLOCKS_OVERRIDE` [(#5032](https://github.com/graphprotocol/graph-node/pull/5032) [(#5117)](https://github.com/graphprotocol/graph-node/pull/5117)
- Initial Starknet Support - Introducing initial Starknet support for graph-node, expanding indexing capabilities to the Starknet ecosystem. The current integration is in its early stages, with notable areas for development including the implementation of trigger filters and data source template support. Future updates will also bring substream support. [(#4895)](https://github.com/graphprotocol/graph-node/pull/4895)
- `endBlock` Feature in Data Sources - This update adds the `endBlock` field for dataSources in subgraph manifest. By setting an `endBlock`, subgraph authors can define the exact block at which a data source will cease processing, ensuring no further triggers are processed beyond this point. [(#4787](https://github.com/graphprotocol/graph-node/pull/4787)
- Autogenerated `Int8` IDs in graph-node - Introduced support for using `Int8` as the ID type for entities, with the added capability to auto-generate these IDs, enhancing flexibility and functionality in entity management. [(#5029)](https://github.com/graphprotocol/graph-node/pull/5029)
- GraphiQL V2 Update - Updated GraphiQL query interface of graph-node to version 2. [(#4677)](https://github.com/graphprotocol/graph-node/pull/4677)
- Sharding Guide for Graph-Node - A new guide has been added to graph-node documentation, explaining how to scale graph-node installations using sharding with multiple Postgres instances. [Sharding Guide](https://github.com/graphprotocol/graph-node/blob/master/docs/sharding.md)
- Per-chain polling interval configuration for RPC Block Ingestors [(#5066)](https://github.com/graphprotocol/graph-node/pull/5066)
- + 5 more
π Bug fixes
- Addressed a bug in the deduplication logic for Cosmos events, ensuring all distinct events are properly indexed and handled, especially when similar but not identical events occur within the same block. [(#5112)](https://github.com/graphprotocol/graph-node/pull/5112)
- Fixed compatibility issues with ElasticSearch 8.X, ensuring proper log functionality. [(#5013)](https://github.com/graphprotocol/graph-node/pull/5013)
- Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources 'from the future' could have been left behind. This release adds a database migration that fixes that. With very unlucky timing this migration might miss some subgraphs, which will later lead to an error `assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block()`. Should that happen, the [migration script](https://github.com/graphprotocol/graph-node/blob/master/store/postgres/migrations/2024-01-05-170000_ds_corruption_fix_up/up.sql) should be rerun against the affected shard. [(#5083)](https://github.com/graphprotocol/graph-node/pull/5083)
- Increased the base backoff time for RPC, enhancing stability and reliability under load. [(#4984)](https://github.com/graphprotocol/graph-node/pull/4984)
- Resolved an issue related to spawning offchain data sources from existing offchain data source mappings. [(#5051)](https://github.com/graphprotocol/graph-node/pull/5051)[(#5092)](https://github.com/graphprotocol/graph-node/pull/5092)
- Resolved an issue where eth-call results for reverted calls were being cached in call cache. [(#4879)](https://github.com/graphprotocol/graph-node/pull/4879)
- Fixed a bug in graphman's index creation to ensure entire String and Bytes columns are indexed rather than just their prefixes, resulting in optimized query performance and accuracy. [(#4995)](https://github.com/graphprotocol/graph-node/pull/4995)
- Adjusted `SubstreamsBlockIngestor` to initiate at the chain's head block instead of starting at block zero when no cursor exists. [(#4951)](https://github.com/graphprotocol/graph-node/pull/4951)
- + 1 more
π¦ Graphman
- Graphman Deploy Command - A new `graphman deploy` command has been introduced, simplifying the process of deploying subgraphs to graph-node. [(#4930)](https://github.com/graphprotocol/graph-node/pull/4930)
- Complete changelog: https://github.com/graphprotocol/graph-node/compare/v0.33.0...v0.34.0
β¨ What's New
- Substreams as Source of Triggers for Subgraphs - This update significantly enhances subgraph functionality by enabling substreams to act as a source of triggers for running subgraph mappings. Developers can now directly run subgraph mappings on the data output from substreams, facilitating a more integrated and efficient workflow.[(#4887)](https://github.com/graphprotocol/graph-node/pull/4887) [(#4916)](https://github.com/graphprotocol/graph-node/pull/4916)
- `indexerHints` in Manifest for Automated Pruning - This update introduces the ability for subgraph authors to specify `indexerHints` with a field `prune` in their manifest, indicating the desired extent of historical block data retention. This feature enables graph-node to automatically prune subgraphs when the stored history exceeds the specified limit, significantly improving query performance. This automated process eliminates the need for manual action by indexers for each subgraph. Indexers can also override user-set historyBlocks with the environment variable `GRAPH_HISTORY_BLOCKS_OVERRIDE` [(#5032](https://github.com/graphprotocol/graph-node/pull/5032) [(#5117)](https://github.com/graphprotocol/graph-node/pull/5117)
- Initial Starknet Support - Introducing initial Starknet support for graph-node, expanding indexing capabilities to the Starknet ecosystem. The current integration is in its early stages, with notable areas for development including the implementation of trigger filters and data source template support. Future updates will also bring substream support. [(#4895)](https://github.com/graphprotocol/graph-node/pull/4895)
- `endBlock` Feature in Data Sources - This update adds the `endBlock` field for dataSources in subgraph manifest. By setting an `endBlock`, subgraph authors can define the exact block at which a data source will cease processing, ensuring no further triggers are processed beyond this point. [(#4787](https://github.com/graphprotocol/graph-node/pull/4787)
- Autogenerated `Int8` IDs in graph-node - Introduced support for using `Int8` as the ID type for entities, with the added capability to auto-generate these IDs, enhancing flexibility and functionality in entity management. [(#5029)](https://github.com/graphprotocol/graph-node/pull/5029)
- GraphiQL V2 Update - Updated GraphiQL query interface of graph-node to version 2. [(#4677)](https://github.com/graphprotocol/graph-node/pull/4677)
- Sharding Guide for Graph-Node - A new guide has been added to graph-node documentation, explaining how to scale graph-node installations using sharding with multiple Postgres instances. [Sharding Guide](https://github.com/graphprotocol/graph-node/blob/master/docs/sharding.md)
- Per-chain polling interval configuration for RPC Block Ingestors [(#5066)](https://github.com/graphprotocol/graph-node/pull/5066)
- + 5 more
π Bug fixes
- Addressed a bug in the deduplication logic for Cosmos events, ensuring all distinct events are properly indexed and handled, especially when similar but not identical events occur within the same block. [(#5112)](https://github.com/graphprotocol/graph-node/pull/5112)
- Fixed compatibility issues with ElasticSearch 8.X, ensuring proper log functionality. [(#5013)](https://github.com/graphprotocol/graph-node/pull/5013)
- Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources 'from the future' could have been left behind. This release adds a database migration that fixes that. With very unlucky timing this migration might miss some subgraphs, which will later lead to an error `assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block()`. Should that happen, the [migration script](https://github.com/graphprotocol/graph-node/blob/master/store/postgres/migrations/2024-01-05-170000_ds_corruption_fix_up/up.sql) should be rerun against the affected shard. [(#5083)](https://github.com/graphprotocol/graph-node/pull/5083)
- Increased the base backoff time for RPC, enhancing stability and reliability under load. [(#4984)](https://github.com/graphprotocol/graph-node/pull/4984)
- Resolved an issue related to spawning offchain data sources from existing offchain data source mappings. [(#5051)](https://github.com/graphprotocol/graph-node/pull/5051)[(#5092)](https://github.com/graphprotocol/graph-node/pull/5092)
- Resolved an issue where eth-call results for reverted calls were being cached in call cache. [(#4879)](https://github.com/graphprotocol/graph-node/pull/4879)
- Fixed a bug in graphman's index creation to ensure entire String and Bytes columns are indexed rather than just their prefixes, resulting in optimized query performance and accuracy. [(#4995)](https://github.com/graphprotocol/graph-node/pull/4995)
- Adjusted `SubstreamsBlockIngestor` to initiate at the chain's head block instead of starting at block zero when no cursor exists. [(#4951)](https://github.com/graphprotocol/graph-node/pull/4951)
- + 1 more
π¦ Graphman
- Graphman Deploy Command - A new `graphman deploy` command has been introduced, simplifying the process of deploying subgraphs to graph-node. [(#4930)](https://github.com/graphprotocol/graph-node/pull/4930)
β¨ What's New
- Arweave file data sources - Arweave file data sources allow subgraph developers to access offchain data from Arweave from within the subgraph mappings.[(#4789)](https://github.com/graphprotocol/graph-node/pull/4789)
- Major performance boost for substreams-based subgraphs - Significant performance improvements have been achieved for substreams-based subgraphs by moving substreams processing to the block stream.[(#4851)](https://github.com/graphprotocol/graph-node/pull/4851)
- Polling block handler - A new block handler filter `polling` for `ethereum` data sources which enables subgraph developers to run a block handler at defined block intervals. This is useful for use cases such as taking periodic snapshots of the contract state.[(#4725)](https://github.com/graphprotocol/graph-node/pull/4725)
- Initialization handler - A new block handler filter `once` for `ethereum` data sources which enables subgraph developers to create a handler which will be called only once before all other handlers run. This configuration allows the subgraph to use the handler as an initialization handler, performing specific tasks at the start of indexing. [(#4725)](https://github.com/graphprotocol/graph-node/pull/4725)
- DataSourceContext in manifest - `DataSourceContext` in Manifest - DataSourceContext can now be defined in the subgraph manifest. It's a free-form map accessible from the mapping. This feature is useful for templating chain-specific data in subgraphs that use the same codebase across multiple chains.[(#4848)](https://github.com/graphprotocol/graph-node/pull/4848)
- `graph-node` version in index node API - The Index Node API now features a new query, Version, which can be used to query the current graph-node version and commit. [(#4852)](https://github.com/graphprotocol/graph-node/pull/4852)
- Added a '`paused`' field to Index Node API, a boolean indicating the subgraphβs pause status. [(#4779)](https://github.com/graphprotocol/graph-node/pull/4779)
- Proof of Indexing logs now include block number [(#4798)](https://github.com/graphprotocol/graph-node/pull/4798)
- + 6 more
π Bug fixes
- Fix for rewinding dynamic data source - Resolved an issue where a rewind would fail to properly remove dynamic data sources when using `graphman rewind`. This has been fixed to ensure correct behavior.[(#4810)](https://github.com/graphprotocol/graph-node/pull/4810)
- Improved Deployment Reliability with Retry Mechanism - A retry feature has been added to the block_pointer_from_number function to enhance the robustness of subgraph deployments. This resolves occasional failures encountered during deployment processes.[(#4812)](https://github.com/graphprotocol/graph-node/pull/4812)
- Fixed Cross-Shard Grafting Issue - Addressed a bug that prevented cross-shard grafting from starting, causing the copy operation to stall at 0% progress. This issue occurred when a new shard was added after the primary shard had already been configured. The fix ensures that foreign tables and schemas are correctly set up in new shards. For existing installations experiencing this issue, it can be resolved by running `graphman database remap`.[(#4845)](https://github.com/graphprotocol/graph-node/pull/4845)
- Fixed a Full-text search regression - Reverted a previous commit (ad1c6ea) that inadvertently limited the number of populated search indexes per entity.[(#4808)](https://github.com/graphprotocol/graph-node/pull/4808)
- Attestable Error for Nested Child Filters - Nested child filter queries now return an attestable `ChildFilterNestingNotSupportedError`, improving error reporting for users.[(#4828)](https://github.com/graphprotocol/graph-node/pull/4828)
π¦ Graphman
- Index on prefixed fields - The graphman index create command now correctly indexes prefixed fields of type String and Bytes for more query-efficient combined indexes. Note: For fields that are references to entities, the behavior may differ. The command may create an index using left(..) when it should index the column directly.
- Partial Indexing for Recent Blocks - The graphman index create command now includes a `--after $recent_block` flag for creating partial indexes focused on recent blocks. This enhances query performance similar to the effects of pruning. Queries using these partial indexes must include a specific clause for optimal performance.[(#4830)](https://github.com/graphprotocol/graph-node/pull/4830)
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/incrypto32/v0.32.0...v0.33.0
β¨ What's New
- Arweave file data sources - Arweave file data sources allow subgraph developers to access offchain data from Arweave from within the subgraph mappings.[(#4789)](https://github.com/graphprotocol/graph-node/pull/4789)
- Major performance boost for substreams-based subgraphs - Significant performance improvements have been achieved for substreams-based subgraphs by moving substreams processing to the block stream.[(#4851)](https://github.com/graphprotocol/graph-node/pull/4851)
- Polling block handler - A new block handler filter `polling` for `ethereum` data sources which enables subgraph developers to run a block handler at defined block intervals. This is useful for use cases such as taking periodic snapshots of the contract state.[(#4725)](https://github.com/graphprotocol/graph-node/pull/4725)
- Initialization handler - A new block handler filter `once` for `ethereum` data sources which enables subgraph developers to create a handle which will be called only once before all other handlers run. This configuration allows the subgraph to use the handler as an initialization handler, performing specific tasks at the start of indexing. [(#4725)](https://github.com/graphprotocol/graph-node/pull/4725)
- DataSourceContext in manifest - `DataSourceContext` in Manifest - DataSourceContext can now be defined in the subgraph manifest. It's a free-form map accessible from the mapping. This feature is useful for templating chain-specific data in subgraphs that use the same codebase across multiple chains.[(#4848)](https://github.com/graphprotocol/graph-node/pull/4848)
- `graph-node` version in index node API - The Index Node API now features a new query, Version, which can be used to query the current graph-node version and commit. [(#4852)](https://github.com/graphprotocol/graph-node/pull/4852)
- Added a '`paused`' field to Index Node API, a boolean indicating the subgraphβs pause status. [(#4779)](https://github.com/graphprotocol/graph-node/pull/4779)
- Proof of Indexing logs now include block number [(#4798)](https://github.com/graphprotocol/graph-node/pull/4798)
- + 6 more
π Bug fixes
- Fix for rewinding dynamic data source - Resolved an issue where a rewind would fail to properly remove dynamic data sources when using `graphman rewind`. This has been fixed to ensure correct behavior.[(#4810)](https://github.com/graphprotocol/graph-node/pull/4810)
- Improved Deployment Reliability with Retry Mechanism - A retry feature has been added to the block_pointer_from_number function to enhance the robustness of subgraph deployments. This resolves occasional failures encountered during deployment processes.[(#4812)](https://github.com/graphprotocol/graph-node/pull/4812)
- Fixed Cross-Shard Grafting Issue - Addressed a bug that prevented cross-shard grafting from starting, causing the copy operation to stall at 0% progress. This issue occurred when a new shard was added after the primary shard had already been configured. The fix ensures that foreign tables and schemas are correctly set up in new shards.[(#4845)](https://github.com/graphprotocol/graph-node/pull/4845)
- Fixed a Full-text search regression - Reverted a previous commit (ad1c6ea) that inadvertently limited the number of populated search indexes per entity.[(#4808)](https://github.com/graphprotocol/graph-node/pull/4808)
- Attestable Error for Nested Child Filters - Nested child filter queries now return an attestable `ChildFilterNestingNotSupportedError`, improving error reporting for users.[(#4828)](https://github.com/graphprotocol/graph-node/pull/4828)
π¦ Graphman
- Index on prefixed fields - The graphman index create command now correctly indexes prefixed fields of type String and Bytes, making combined indexes more query-efficient.
- Partial Indexing for Recent Blocks - The graphman index create command now includes a `--after $recent_block` flag for creating partial indexes focused on recent blocks. This enhances query performance similar to the effects of pruning. Queries using these partial indexes must include a specific clause for optimal performance.[(#4830)](https://github.com/graphprotocol/graph-node/pull/4830)
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.32.0...b1502a426227145d890df87483c3890d66ecf2db
β¨ What's New
- Derived fields getter: Derived fields can now be accessed from within the mapping code during indexing. ([#4434](https://github.com/graphprotocol/graph-node/pull/4434))
- Sorting interfaces by child entity: Interfaces can now be sorted by non-derived child entities. ([#4058](https://github.com/graphprotocol/graph-node/pull/4058))
- File data sources can now be spawned from handlers of other file data sources: This enables the use of file data sources for scenarios where a file data source needs to be spawned from another one. One practical application of this feature is in handling NFT metadata. In such cases, the metadata itself is stored as a file on IPFS and contains embedded IPFS CID for the actual file for the NFT. ([#4713](https://github.com/graphprotocol/graph-node/pull/4713))
- Allow redeployment of grafted subgraphs even when graft_base is not available: This will allow renaming of already synced grafted subgraphs even when the graft base is not available, which previously failed due to `graft-base` validation errors. ([#4695](https://github.com/graphprotocol/graph-node/pull/4695))
- `history_blocks` is now available in the index-node API. ([#4662](https://github.com/graphprotocol/graph-node/pull/4662))
- Added a new `subgraph features` table in `primary` to easily track information like `apiVersion`, `specVersion`, `features`, and data source kinds used by subgraphs. ([#4679](https://github.com/graphprotocol/graph-node/pull/4679))
- `subgraphFeatures` endpoint now includes data from `subgraph_features` table.
- `ens_name_by_hash` is now undeprecated: This reintroduces support for fetching ENS names by their hash, dependent on the availability of the underlying [Rainbow Table](https://github.com/graphprotocol/ens-rainbow) ([#4751](https://github.com/graphprotocol/graph-node/pull/4751)).
- + 5 more
π₯ Breaking changes
- Duplicate provider labels are not allowed in graph-node config anymore
π Bug fixes
- Fixed `PublicProofsOfIndexing` returning the error `Null value resolved for non-null field proofOfIndexing` when fetching POIs for blocks that are not in the cache ([#4768](https://github.com/graphprotocol/graph-node/pull/4768))
- Fixed an issue where Block stream would fail when switching back to an RPC-based block ingestor from a Firehose ingestor. ([#4790](https://github.com/graphprotocol/graph-node/pull/4790))
- Fixed an issue where derived loaders were not working with entities with Bytes as IDs ([#4773](https://github.com/graphprotocol/graph-node/pull/4773))
- Firehose connection test now retries for 30 secs before setting the provider status to `Broken` ([#4754](https://github.com/graphprotocol/graph-node/pull/4754))
- Fixed the `nonFatalErrors` field not populating in the index node API. ([#4615](https://github.com/graphprotocol/graph-node/pull/4615))
- Fixed `graph-node` panicking on the first startup when both Firehose and RPC providers are configured together. ([#4680](https://github.com/graphprotocol/graph-node/pull/4680))
- Fixed block ingestor failing to startup with the error `net version for chain mainnet has changed from 0 to 1` when switching from Firehose to an RPC provider. ([#4692](https://github.com/graphprotocol/graph-node/pull/4692))
- Fixed Firehose endpoints getting rate-limited due to duplicated providers during connection pool initialization. ([#4778](https://github.com/graphprotocol/graph-node/pull/4778))
- + 1 more
π¦ Graphman
- Added two new `graphman` commands `pause` and `resume`: Instead of reassigning to a non-existent node these commands can now be used for pausing and resuming subgraphs. ([#4642](https://github.com/graphprotocol/graph-node/pull/4642))
- Added a new `graphman` command `restart` to restart a subgraph. ([#4742](https://github.com/graphprotocol/graph-node/pull/4742))
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.31.0...2891a76ef30194123c60578c2dc4395d243d05d3
β¨ What's New
- Derived fields getter: Derived fields can now be accessed from within the mapping code during indexing. ([#4434](https://github.com/graphprotocol/graph-node/pull/4434))
- Sorting interfaces by child entity: Interfaces can now be sorted by non-derived child entities. ([#4058](https://github.com/graphprotocol/graph-node/pull/4058))
- File data sources can now be spawned from handlers of other file data sources: This enables the use of file data sources for scenarios where a file data source needs to be spawned from another one. One practical application of this feature is in handling NFT metadata. In such cases, the metadata itself is stored as a file on IPFS and contains embedded IPFS CID for the actual file for the NFT. ([#4713](https://github.com/graphprotocol/graph-node/pull/4713))
- Allow redeployment of grafted subgraphs even when graft_base is not available: This will allow renaming of already synced grafted subgraphs even when the graft base is not available, which previously failed due to `graft-base` validation errors. ([#4695](https://github.com/graphprotocol/graph-node/pull/4695))
- `history_blocks` is now available in the index-node API. ([#4662](https://github.com/graphprotocol/graph-node/pull/4662))
- Added a new `subgraph features` table in `primary` to easily track information like `apiVersion`, `specVersion`, `features`, and data source kinds used by subgraphs. ([#4679](https://github.com/graphprotocol/graph-node/pull/4679))
- `subgraphFeatures` endpoint now includes data from `subgraph_features` table.
- `ens_name_by_hash` is now undeprecated: This reintroduces support for fetching ENS names by their hash, dependent on the availability of the underlying [Rainbow Table](https://github.com/graphprotocol/ens-rainbow) ([#4751](https://github.com/graphprotocol/graph-node/pull/4751)).
- + 5 more
π Bug fixes
- Fixed `PublicProofsOfIndexing` returning the error `Null value resolved for non-null field proofOfIndexing` when fetching POIs for blocks that are not in the cache ([#4768](https://github.com/graphprotocol/graph-node/pull/4768))
- Fixed an issue where Block stream would fail when switching back to an RPC-based block ingestor from a Firehose ingestor. ([#4790](https://github.com/graphprotocol/graph-node/pull/4790))
- Fixed an issue where derived loaders were not working with entities with Bytes as IDs ([#4773](https://github.com/graphprotocol/graph-node/pull/4773))
- Firehose connection test now retries for 30 secs before setting the provider status to `Broken` ([#4754](https://github.com/graphprotocol/graph-node/pull/4754))
- Fixed the `nonFatalErrors` field not populating in the index node API. ([#4615](https://github.com/graphprotocol/graph-node/pull/4615))
- Fixed `graph-node` panicking on the first startup when both Firehose and RPC providers are configured together. ([#4680](https://github.com/graphprotocol/graph-node/pull/4680))
- Fixed block ingestor failing to startup with the error `net version for chain mainnet has changed from 0 to 1` when switching from Firehose to an RPC provider. ([#4692](https://github.com/graphprotocol/graph-node/pull/4692))
- Fixed Firehose endpoints getting rate-limited due to duplicated providers during connection pool initialization. ([#4778](https://github.com/graphprotocol/graph-node/pull/4778))
π¦ Graphman
- Added two new `graphman` commands `pause` and `resume`: Instead of reassigning to a non-existent node these commands can now be used for pausing and resuming subgraphs. ([#4642](https://github.com/graphprotocol/graph-node/pull/4642))
- Added a new `graphman` command `restart` to restart a subgraph. ([#4742](https://github.com/graphprotocol/graph-node/pull/4742))
- Full Changelog: https://github.com/graphprotocol/graph-node/compare/v0.31.0...c350e4f35c49bcf8a8b521851f790234ba2c0295
`v0.31.0-rc.1` is the second release candidate of the upcoming `v0.31.0`. ``` $ docker pull graphprotocol/graph-node:v0.31.0-rc.1 ``` Edit 2023-06-08: `v0.31.0-rc.1` has been promoted to [`v0.31.0`](https://github.com/graphprotocol/graph-node/releases/tag/v0.31.0).
β¨ What's new
- Fulltext searches can now be combined with `where` filtering, further narrowing down search results. [#4442](https://github.com/graphprotocol/graph-node/pull/4442)
- Derived fields loader support. This feature requires subgraph authors to upgrade `graph-cli` to v0.51.0. [#4434](https://github.com/graphprotocol/graph-node/pull/4434), [`graph-cli` release notes](https://github.com/graphprotocol/graph-tooling/releases/tag/%40graphprotocol%2Fgraph-cli%400.51.0)
- Tweaked how RPC provider limiting rules are interpreted from configurations. In particular, node IDs that don't match any rules of a provider won't have access to said provider instead of having access to it for an unlimited number of subgraphs. Read the [docs](https://github.com/graphprotocol/graph-node/pull/4353/files) for more information. [#4353](https://github.com/graphprotocol/graph-node/pull/4353)
- Introduced WASM host function `store.get_in_block`, which is a much faster variant of `store.get` limited to entities created or updated in the current block. [#4540](https://github.com/graphprotocol/graph-node/pull/4540)
- The `subgraph_deploy` JSON-RPC method now accepts a `history_blocks` parameter, which indexers can use to set default amounts of history to keep. [#4564](https://github.com/graphprotocol/graph-node/pull/4564)
- IPFS requests for polling file data sources are not throttled anymore (also known as concurrency or burst limiting), only rate-limited. [#4570](https://github.com/graphprotocol/graph-node/pull/4570), [#4649](https://github.com/graphprotocol/graph-node/pull/4649)
- Exponential requests backoff when retrying failed subgraphs is now "jittered", smoothing out request spikes. [#4476](https://github.com/graphprotocol/graph-node/pull/4476)
- Indexers can use thee new feature `GRAPH_EXPERIMENTAL_SUBGRAPH_SETTINGS` to define matching rules on subgraph names for reducing amounts of history stored. [#4633](https://github.com/graphprotocol/graph-node/pull/4633)
- + 7 more
β‘ Performance improvements
- Batched writes: write operations to the database when indexing subgraphs are now accumulated into in-memory batches before persisting them, resulting in noticeably faster indexing and lower database CPU load. In case of unexpected issues, this feature can be turned off setting `GRAPH_STORE_WRITE_BATCH_SIZE=0`. [#4606](https://github.com/graphprotocol/graph-node/pull/4606), [#4632](https://github.com/graphprotocol/graph-node/pull/4632)
- The entity cache that `graph-node` keeps around is much more efficient, meaning more cache entries fit in the same amount of memory resulting in a performance increase under a wide range of workloads. [#4485](https://github.com/graphprotocol/graph-node/pull/4485), [#4624](https://github.com/graphprotocol/graph-node/pull/4624)
- Indexing speed improvements for subgraphs with more than 10k data sources. [#4631](https://github.com/graphprotocol/graph-node/pull/4631)
- BRIN indexes in PostgreSQL now use `..._minmax_multi_ops` instead of `_minmax_ops` if available (PostgreSQL 14+), often resulting in more selective indexes. [#4629](https://github.com/graphprotocol/graph-node/pull/4629)
- GIN indexes are not created anymore for numeric arrays. [#4628](https://github.com/graphprotocol/graph-node/pull/4628)
π¦ Substreams
- The substreams protocol has been updated to `sf.substreams.rpc.v2.Stream/Blocks`. [#4556](https://github.com/graphprotocol/graph-node/pull/4556)
- Added support for adapter balancing a.k.a. provider failover to substreams. [#4578](https://github.com/graphprotcol/graph-node/pull/4578)
- Fixed faulty `startBlock` selection logic in substreams. [#4463](https://github.com/graphprotocol/graph-node/pull/4463)
- Several bug fixes were applied after breakage caused by an internal refactor of how entities are stored in-memory. [#4604](https://github.com/graphprotocol/graph-node/pull/4604), [#4572](https://github.com/graphprotocol/graph-node/pull/4572), [#4612](https://github.com/graphprotocol/graph-node/pull/4612), [#4664](https://github.com/graphprotocol/graph-node/pull/4664)
π Bug fixes
- Fixed a bug that would cause subgraphs to fail with a `subgraph writer poisoned by previous error` message following certain database errors. [#4533](https://github.com/graphprotocol/graph-node/pull/4533)
- Fixed a bug that would cause subgraphs to fail with a `store error: no connection to the server` message when database connection e.g. gets killed. [#4435](https://github.com/graphprotocol/graph-node/pull/4435)
- The `subgraph_reassign` JSON-RPC method doesn't fail anymore when multiple deployment copies are found: only the active copy is reassigned, the others are ignored. [#4395](https://github.com/graphprotocol/graph-node/pull/4395)
- Fixed a bug that would cause `on_sync` handlers on copied deployments to fail with the message `Subgraph instance failed to run: deployment not found [...]`. [#4396](https://github.com/graphprotocol/graph-node/pull/4396)
- Fixed a bug that would cause the copying or grafting of a subgraph while pruning it to incorrectly set `earliest_block` in the destination deployment. [#4502](https://github.com/graphprotocol/graph-node/pull/4502)
- Handler timeouts would sometimes be reported as deterministic errors with the error message `Subgraph instance failed to run: Failed to call 'asc_type_id' with [...] wasm backtrace [...]`; this error is now nondeterministic and recoverable. [#4475](https://github.com/graphprotocol/graph-node/pull/4475)
- Fixed faulty exponential request backoff behavior after many minutes of failed requests, caused by an overflow. [#4421](https://github.com/graphprotocol/graph-node/pull/4421)
- `json.fromBytes` and all `BigInt` operations now require more gas, protecting against malicious subgraphs. [#4594](https://github.com/graphprotocol/graph-node/pull/4594), [#4595](https://github.com/graphprotocol/graph-node/pull/4595)
- + 1 more
π¦ Graphman
- `graphman rewind` now requires `block-number` and `block-hash` to be passed as flags instead of arguments. [#4400](https://github.com/graphprotocol/graph-node/pull/4400)
- You can now use the new flag `--start-block` in `graphman rewind` to rewind a subgraph to the `startBlock` set in manifest, or to the genesis block if no `startBlock` is set. [#4400](https://github.com/graphprotocol/graph-node/pull/4400)
- The behavior for `graphman prune` has changed: running just `graphman prune` will mark the subgraph for ongoing pruning in addition to performing an initial pruning. To avoid ongoing pruning, use `graphman prune --once` ([docs](./docs/implementation/pruning.md)). [#4429](https://github.com/graphprotocol/graph-node/pull/4429)
- The env. var. `GRAPH_STORE_HISTORY_COPY_THRESHOLD` βwhich serves as a configuration setting for `graphman prune`β has been renamed to `GRAPH_STORE_HISTORY_REBUILD_THRESHOLD`. [#4505](https://github.com/graphprotocol/graph-node/pull/4505)
- You can now list all existing deployments via `graphman info --all`. [#4347](https://github.com/graphprotocol/graph-node/pull/4347)
- The command `graphman chain call-cache remove` now requires `--remove-entire-cache` as an explicit flag, protecting against accidental destructive command invocations. [#4397](https://github.com/graphprotocol/graph-node/pull/4397)
- `graphman copy create` accepts two new flags, `--activate` and `--replace`, which make moving of subgraphs across shards much easier. [#4374](https://github.com/graphprotocol/graph-node/pull/4374)
- The log level for `graphman` is now set via `GRAPHMAN_LOG` or command line instead of `GRAPH_LOG`. [#4462](https://github.com/graphprotocol/graph-node/pull/4462)
- + 1 more
π¦ Metrics and logging
- Subgraph syncing time metric `deployment_sync_secs` now stops updating once the subgraph has synced. [#4489](https://github.com/graphprotocol/graph-node/pull/4489)
- New `endpoint_request` metric to track error rates of different providers. [#4490](https://github.com/graphprotocol/graph-node/pull/4490), [#4504](https://github.com/graphprotocol/graph-node/pull/4504), [#4430](https://github.com/graphprotocol/graph-node/pull/4430)
- New metrics `chain_head_cache_num_blocks`, `chain_head_cache_oldest_block`, `chain_head_cache_latest_block`, `chain_head_cache_hits`, and `chain_head_cache_misses` to monitor the effectiveness of `graph-node`'s in-memory chain head caches. [#4440](https://github.com/graphprotocol/graph-node/pull/4440)
- The subgraph error message `store error: Failed to remove entities` is now more detailed and contains more useful information. [#4367](https://github.com/graphprotocol/graph-node/pull/4367)
- `eth_call` logs now include the provider string. [#4548](https://github.com/graphprotocol/graph-node/pull/4548)
- Tweaks and small changes to log messages when resolving data sources, mappings, and manifests. [#4399](https://github.com/graphprotocol/graph-node/pull/4399)
- `FirehoseBlockStream` and `FirehoseBlockIngestor` now log adapter names. [#4411](https://github.com/graphprotocol/graph-node/pull/4411)
- Runtime panics now provide more contextual information, instead of just `oneshot cancelled`. [#4625](https://github.com/graphprotocol/graph-node/pull/4625)
- + 9 more
`v0.31.0-rc.0` is the first release candidate of the upcoming `v0.31.0`. This release candidate is intended for testnet use only. ``` $ docker pull graphprotocol/graph-node:v0.31.0-rc.0 ``` Edit 2023-06-08: `v0.31.0-rc.0` is obsolete; [`v0.31.0-rc.1`](https://github.com/graphprotocol/graph-node/releases/tag/v0.31.0-rc.1) is the latest release candidate.
