* Fixed NullReferenceException: Addressed a crash in JwtSecurityTokenHandler under .NET 9 caused by internal changes in cryptographic primitives and claim handling.
* Migrated JWT Handling: Updated Jwt.CreateToken to use JsonWebTokenHandler with SecurityTokenDescriptor.
The method now returns a string directly (formerly JwtSecurityToken).
* Configured AddJwtBearer with UseSecurityTokenValidators = false to ensure end-to-end consistency using JsonWebTokenHandler.
* Updated AutoMapper: Adjusted the instantiation signature to include a null logger following the upgrade from v13 to v15 (required to patch a CodeQL security vulnerability). Resolved high-severity DoS vulnerability GHSA-rvv3-g6hj-g44x (uncontrolled recursion) and fixed the new instantiation signature requiring a null logger.
* Resolve secrets from file topology where file name is key and trimmed
file contents are value, compatible with Docker Secrets tmpfs mounting.
* Support [@Key@] token pattern inside Environment Variable strings,
allowing verbose connection strings to live outside declared secrets.
* SecretsPath configurable via environment variable or appSettings,
falling back to working directory for local development convenience.
* Secrets processing occurs at startup before logging, with diagnostics
written to STDOUT.
* Tokenisation extracted into standalone library and included in Dockerfile
* Documentation added covering token pattern and Docker Secrets compatibility.
* Replaced the StackOverflow Redis client in the Cache library with the Resilient Redis Library.
* Improved policy wrappers by introducing two distinct policies: one for idempotent operations and one for increment operations. Added a circuit breaker to the Redis Resiliency layer to prevent thundering herd on re-instantiation, along with shared state tracking for known-down Redis instances.
* Updated the Redis resilient command wrapper to proxy all commands through the appropriate policy (e.g. based on idempotency).
* Implemented certain other Redis policy state detection.
* Implemented a wrapper library for Redis and Postgres clients using Polly retry to enhance durability during failover. Removed all direct client references across projects, replacing them with the new resilient libraries. Bulk changes were largely confined to the database context constructor, given the application's broad use of the Repository pattern and LINQ2DB.
* Implemented a WAL log in Redis that stores JSON payloads pending commit to the Archive table as part of an asynchronous process. The WAL entry is removed once the commit completes. During graceful drain on SIGTERM, this provides an observability and recovery mechanism for transactions lost due to host failure or fatal termination.
* Implemented a read-only reporting connection for all case management query functions and other backend functions with significant read overhead or security implications, enabling targeting of read replicas under read-only database users. Dynamic query functions have been handled carefully to ensure parameterised variables are always used.
* Implemented a `MigrationConnectionString` environment variable to support a dedicated migration user, allowing the application and reporting to run under minimal permissions while still supporting the online upgrade process used in Jube Herd (Docker Swarm).
* Added evaluation counters to Gateway and Activation Rule Counters, and ensured they are displayed in the Gateway Rules, Activation Rules, and Activation Rules pages — an important missing observability feature for gradual rule rollout assessment prior to taking decline or case creation steps. Also introduced model total response time counters throughout the invocation pipeline and updated the administration page accordingly. Made all response times throughout the system record in microseconds for consistency.
* Updated `CookieOptions` to include the `Secure` flag having been flagged on CodeQL.
* Fixed major bugs in fan-out queries across all Case Management entities caused by the recently released permissions functionality, where entities were duplicated when more than one user was present.
* Fixed a long-standing issue with performance counter increments to use `Interlocked`, or custom locks in the case of date-based counters.
* Fixed a bug in preservation import introduced during the `CacheIndexId` functionality, which failed due to a nested transaction. The import now bypasses `CacheIndexId` rekeying and reads it directly from the `.jemp` file, which was itself the correct original behaviour.
* Fixed Payload Responses not being included in the response payload when response payload was selected in Request XPath.
* Fixed the Report Table switch in Request XPath having no effect and not writing to the `ArchiveKeys` table.
* Fixed the inability to edit Case Workflows in the user interface because of a minor JS typo in the key name.
* Refactored notification dispatch to use async methods for all database calls.
* Refactored the Postgres Query class to make better use of `using`/dispose patterns, moved connection creation to the constructor, and ensured consistent disposal. Extended the use of `using` dispose throughout all LINQ2DB queries outside the Repository pattern.
* Some refactoring to use byte arrays rather than `MemoryStream` in serialisation of JSON Archive and response payloads. Eliminated duplication in Archive payload creation for case creation, given the payload now also exists in the WAL, and is created during invocation (not asynchronously as was the case).
* Updated the composite index for `AddArchiveKeyTableIndex` in Baseline to use covering indexes ONLY, avoiding Postgres page lookups (noting this table exists solely as an access path to the Archive table via predication and join on `EntityAnalysisModelInstanceEntryGuid`). IMPORTANT NOTE: No migration provided for existing data. Any such index will need to be rebuilt manually to obtain this benefit.
* Updated the completions controller and query to order fields by name ascending, with new repository methods to apply ordering server-side.
* Updated the date format for Created Dates in the user interface to a more verbose and readable format, consistent with the rest of the system.
* Removed legacy billing aggregation counter for response elevation, which is no longer a relevant concept. Renamed remaining billing-related counter fields for clarity.
* Updated Dockerfile for the new resilience projects.
* Fixed typo in TTL Counter Resolution page where hours was duplicated instead of minutes.
* Implemented RedisBackplane environment variable to support SignalR with Redis in clustered environments.
* Fixed loss of password, password expiry, and password created date during User Administration updates; added empty password check in authentication controller to raise an exception treated as user not found; fixed inability to change password on expiry; fixed password expiry date evaluation bug; moved increment password failure to capture all code paths.
* Updated ExhaustiveSearchInstanceTrialInstanceVariable repository to use logical deletes instead of physical deletes, which are unsupported by the database user; updated MockArchive loading to occur only once since the data is static.
* Updated default Log4NetLogLevel environment variable to WARN and updated documentation accordingly.
* Fixed missing reference for Archive Keys during payload creation, as while they were being populated, the reference to it was missing, so they did not get stored.
* Added `Cache` flag to `EntityAnalysisRequestXPath`, updating POCOs, DTOs, and synchronisation mappings. Included migrations and updated pages and JS for new fields.
* Updated the Completions Endpoint to only return values if Cache is enabled in the EntityAnalysisModelRequestXPath for EntityAnalysisModelAbstractionRule configuration.
* Updated the parser to ignore fields if the parse processing type is Abstraction Rule, so to ensure that an Abstraction Rule does not compile anywhere, in the controller or syncronisation, if it trys to use a key that exists but is marked as Cache.
* Created migrations for the cache flag and added indexes to cache key columns.
* Implemented a new serialisation pattern for message pack payloads to reduce byte storage of names, replacing them with a `CacheIndexId` serialised as a single byte.
* Implemented indexing logic for `CacheIndexId`, which increments within a transaction per `EntityAnalysisModel`. Created `CacheIndexId` values for legacy data.
* Created migrations for legacy Redis cache payload data to adopt the new `CacheIndexId` format. Added a unique composite index to enforce sequencing and eliminate conflict risks.
* Implemented a new LRU cache warming strategy that populates the LRU cache based on journal usage. During warming, cache entries are populated in order of most recently used journals, exiting when byte limits are reached. This replaces a full cursor scan of payload data, reducing container startup time and mitigating Redis herd-stampede risks. Each journal recall writes to a sorted set (`LruJournals`) that maintains an ordered activity list for cache population. On payload entry removal, the corresponding journal is checked and the `LruJournal` key is removed if no further data exists. Added a `SortedSetLruJournalRemove` enum for task tracking. Implemented an `IsFull` method on the local LRU cache to exit warming once full.
* Updated TTL Counters to process `double` values instead of `int` to support summing of integer and float payload types, making Volume and Velocity rules available to TTL Counters and long-term state. Updated POCOs, DTOs, and synchronisation mappings, applied logic in invocation processes, and included migrations. Updated pages and JS for new fields.
* Updated TTL Counters to specify date resolution for TTL Counter entries, reducing memory footprint by allowing aggregation at broader date roundings (e.g. for long-term state, a daily entry expiry is sufficient). Updated VB.NET code wrappers for the new generic.
* Updated TTL Counters to support an `EnableSum` flag, which uses a `TtlCounterDataValue` field to specify the value to sum. Implemented a dropdown in the page and JS that returns only integer and float data types for selection.
* Updated the efficient no-boxing dictionary to instantiate on the basis of generic type, enabling use with both `int` (message pack) and `string` (core processing and legacy payload migrations). This touched a large number of files and is the primary reason for the size of this commit. Updated the dictionary to support addition of internal values via the interface for serialisation and mapping. Updated JSON serialisers to support the generic.
* Implemented invocation mapping logic for insert and recall between the indexed Redis payload and the string-keyed payload required for processing.
* Bumped the message pack envelope version flag to version 2 to reflect the new payload formats (noting that all data has been migrated and version 1 no longer exists in Redis).
* Updated documentation to cover `LruJournals` in the Redis section, the `Cache` field on the `EntityAnalysisModelsRequestXPath` page, and the `ResolutionInterval` and `TtlCounterDataValue` fields on the `EntityAnalysisTtlCounters` page.
* Pushed Case Workflow Display HTML rendering logic down into a controller for improved security and consistent tokenisation of the Jube token format against context. Included a migration to update the default demo entry to use proper tokenisation. Updated the repository to support fetching CaseWorkflowDisplay with appropriate security.
* Implemented improved tokenisation of the EntityAnalysisModelInstanceEntryPayload model via a common extension method on EntityAnalysisModelInstanceEntryPayload, used consistently across the system for serialisation and storage.
* Included a migration to update the tokenisation format for full CaseWorkflowDisplay support.
* Fixed an issue where CaseWorkflowFilter was incorrectly filtering on CaseWorkflowStatus.Guid instead of CaseWorkflowStatus.Name, causing the GUID to be returned rather than the display name on the template/demo model. (The remainder was fixed in GitHub issue #6.) Included a migration with a corrective SQL statement.
* Fixed an issue where tokens were not being stored alongside CaseWorkflowFilter definition configuration. It now behaves identically to CaseSearch. Note: migrating missing tokens is not feasible as it requires JS-side parsing via JQueryBuilder.
* Reduced the size of the CaseWorkflowForm invocation to include only the required fields given the new POST body context being rendered server side. Improved API serialisation by using a dictionary for form payload keys and values.
* Consolidated JSON serialisation settings into DI and updated usages throughout.
* Updated EntityAnalysisModelInstanceEntryPayload write serialisers to handle deserialisation, which is required to support tokenisation of new Case Automation POST bodies.
* Created an extension method on the EntityAnalysisModelInstanceEntryPayload to centralise tokenisation of strings across Activation and Case Automation on a consistent basis. Simplified Case Notification and Case HTTP dispatch by removing tokenisation logic from those classes.
* Removed a redundant field from the CaseWorkflowFormEntry POCO (database left unchanged for now).
* Fixed a bug in Case session SQL rebuilding where the rebuild date was not being correctly checked for null or default value, which caused a failure in the last migration.
* Made minor modifications to the Create Case process and method signature to skip serialisation when the payload object required for tokenisation is already available on the thread (e.g. during synchronous reprocessing).
* Updated Case Workflow Macro processing to fetch the archive payload for the case from the database, improving security of that invocation endpoint.
* Removed what appeared to be a duplicate EntityAnalysisModelInstanceEntryPayload model introduced during refactoring.
* Fixed a but in Case Worfkflow Display Role assigment owing to volatile Guid in its update.
* Introduced `PartialResponseMessageSerialisation to enable full serialization responses during development and testing.
* Added PartialResponseMessageSerialisation environment variable to relevant documentation.
* Relocated EntityAnalysisModelAdaptations to its proper location within the context model.
* Implemented a conditional serializer that dynamically builds JObject properties based on active Response Payload switches. When no switches are enabled, the array is excluded entirely from the response.
* Response payloads now default to partial serialization (PartialResponseMessageSerialisation defaults to true).
* Renamed methods in the BuildJsonResponses given new functionality.
* Updated documentation to include Inline Script events and Activation Rule Chaining concepts.
* Included Activation Rules in Activation Completions to make them available.
* Modified activation rule signatures to include match keys for facilitation.
* Updated Gateway Rule Priority to support double type, including changes in migrations and POCOs.
* Updated tree to ensure entities are listed by Priority when Priority exists in the entity.
* Included Priority in Activation Rule user interface, POCOs, and migrations; ensured invocation and synchronization preserve order for evaluation sequence to support rule chaining.
* Updated completions to support Activation Rules and Activation Rule chaining.
* Improved delegate compilation to avoid reflection during invocation; delegates are now compiled during synchronization, eliminating reflection usage during invocation for real-time processing.
* Created event attributes to direct matching of Inline Scripts, including matching properties given event context (to restrict to a single Activation Rule, for example).
* Separated ExecuteAsync attributes from property attributes in the solution for better organization.
* Mapped Inline Script property and method attributes during synchronization to avoid reflection during invocation.
* Removed parallel for loops and locks in Activation Rule processing to support chaining.
* Modified reflection wrapper for Inline Scripts to eliminate reflection by compiling delegates during synchronization.
* Minor refactoring and variable renaming.
* Updated Inline Script interface to mandate bool return value in ExecuteAsync method.
* Updated rule parser to support evaluation of exists activation rule match for chaining support.
* Updated sandbox code for new Inline Script format.
* Corrected some spelling in documentation.
* Extracted Case Notification and HTTP Web Hook functionality into a shared Jube.Case library for use across Jube.App and Jube.Engine.
* Added the new Jube.Case library to the Jube.App Docker Compose configuration.
* Cleaned up classes moved to Jube.Case, converting to static classes or methods where appropriate.
* Implemented Case Status automation within the asynchronous background Case Creation process, including upward status classification.
* Added role‑based data isolation across all entities related to Case Workflows and Visualisation.
* Introduced a common UI JavaScript widget for assigning roles to supported entities; updated existing JS components and generic CRUD logic to integrate the new widget.
* Added role‑permission tables and corresponding migrations, POCOs, repositories, controllers, DTOs, and validations for all in‑scope entities.
* Updated all end‑user queries to include joins through role allocations, enforcing data isolation for both Case Management and Visualisation.
* Included role allocations in preservation/backup logic to ensure they can be exported and restored correctly.
* Added documentation describing the isolation model and validation behaviour for Case Management and Visualisation.
* Renamed various architectural methods where ActiveOnly semantics now also imply role‑allocation filtering.
* Improved “no data found” handling on the case page to avoid appearing as though data is still loading.
* Updated the case query compilation routine to enforce role‑based isolation.
* Forced session cache invalidation via migrations to rebuild SQL with new filtering rules.
* Fixed audit record mapping error in datasource update that was causing failures.
* Refactored fill series to use transactional column replacement after query introspection, with new repository delete methods.
* Added backend parameter validation and default handling for visualization requests.
* Fixed JavaScript parameter serialization issues and set explicit Content-Type headers for proper Web API deserialization.
* Refactored visualization recall controller for cleaner code and moved mutable audit logic from query to controller.
* Refactored Archive SQL generation and fixed parse compilation. Separated the Archive query into a SELECT section (handling Request XPath and Inline Script properties), and a block for table definitions and predication.
* Updated the structure to allow select components to be built across multiple synchronization steps.
* Fixed compilation error for VB.net fragment by correcting the fallback value in parse from .AsString() to AsString() to resolve a syntax-induced build failure.
* Updated the Activation.js to include an offset to default to last minute for replay in the user interface.
* Fixed a missing reference which meant that the enqueue to activation watcher concurrent queue was not being picked up by the background process responsible for actual database insert, although it was running.
* Refactored the background persistence process to individial inserts rather than bulk copy, as activation load should never be so significant that it needs a bulk approach, while instead being fairly time sensitive.
* Renamed GetModelFieldByEntityAnalysisModelIdParseTypeIdQuery to GetEntityAnalysisModelFieldByEntityAnalysisModelIdParseTypeIdQuery to align with database model naming.
* Fixed bugs where Inline Scripts were not being matched correctly in Completions and Soft Parsing because EntityAnalysisModelInlineScript.Id was being used instead of EntityAnalysisInlineScript.Id (affecting three areas).
* Swapped Latitude and Longitude matching in Activation Watcher processing for Activation Rules.
* Updated the Sandbox code for Inline Scripts to be less brittle and a more reliable example for client use.
* Updated documentation for the revised Inline Script approach, aligning with context, strategy, and pipeline patterns. Inline Scripts now use a transient instance model; dependencies are assumed to exist in the context, and state is not supported.
* Added a syntax tree helper parser for VB.net and C# in the data layer, enhancing queries for request XPath fields and Inline Script properties in the UI. The syntax tree helper is a Roslyn-based parser for Inline Scripts that extracts public properties without a full compile, enabling property access in the UI for rule writing and abstraction rules.
* Inline Script properties fully supported in parser, completions for coder and builder, and abstraction rule grouping keys, with search keys synchronized via the `SearchKey` attribute.
* Refactored Inline Script methdology to be a transient instance class, merging payload values from public properties via reflection, removing the need to manually add properties.
* Added migrations to include a language identifier for Inline Scripts.
* Inline Scripts now fully support C# and Visual Basic (VB.net).
* Removed `ClassName` and `MethodName` from the `EntityAnalysisInlineScript` table, now derived from the IInlineScript interface.
* Updated Activation Rule Library to fall back to Inline Script properties if Latitude and Longitude are not specified in the Request XPath.
* Moved Inline Script execution to asynchronous methods to avoid blocking IO during potential remote calls; updated reflection helpers accordingly.
* Renamed variables to better match POCO conventions, as came across them.
* Updated assembly loading for Inline Scripts to include all assemblies in the Jube.App context.
* Optimized Inline Script synchronization to reduce unnecessary reflection during invocation.
* Created Jube.Sandbox for testing and deploying Inline Scripts outside the main application.
* Updated Inline Script OTP demo code new Inline Script Interface signature.
* Added name duplication checks to validations for most endpoints invoked via the CRUD.js functionality (applies to most configuration pages).
* Improved user interface rendering of backend validation errors, including styled lists and highlighted form elements on validation error.
* Implemented tree data filtering in configuration tree to speed up exploration.
