For Quine, a recipe is a document that contains all of the information necessary for Quine to execute any batch or streaming data process. Quine recipes are written in
yaml and built from components including:
- Ingest Streams to read streaming data from sources and update graph data
- Standing Queries to transform graph data, and to produce aggregates and other outputs
- Cypher expressions to implement graph operations such as querying and updating data
- Exploration UI configuration to customize the web user interface for the use-case that is the subject of the recipe
Recipes enable you to quickly iterate in Quine and to share what you’ve built with others so they can reproduce, explore and expand upon a solution.
Consider writing a recipe:
- When you are actively developing an event streaming solution
- To preserve a solution whenever a development milestone is achieved
- To configure the visual aspects of the Quine Exploration UI
- To store
sample queriesthat aide in graph analysis
- To share your solution with collaborators or the open source community
- When interacting with Quine support
A recipe is stored in a single
YAML text file. The file contains a single object with the following attributes:
version: 1 title: recipe Title contributor: https://github.com/example-user summary: "" description: "" ingestStreams:  standingQueries:  nodeAppearances:  quickQueries:  sampleQueries:  statusQuery: null
Each configuration object is defined by its corresponding API entity in the REST API. Follow the links in the table below for details regarding each attribute.
||Integer||The recipe schema version, right now the only supported value is 1|
||String||Identifies the recipe|
||String||URL to social profile of the person or organization responsible for this recipe|
||String||Brief information about this recipe|
||Text Block||Long form description about this recipe|
||Array of IngestStream objects||Define how data is read from data sources, transformed, and loaded into the graph|
||Array of StandingQuery objects||Define both sub-graph patterns for Quine to match and subsequent output actions|
||Array of NodeAppearance objects||Customize node appearance in the exploration UI|
||Array of QuickQuery objects||Add queries to node context menus in the exploration UI|
||Array of SampleQuery objects||Customize sample queries listed in the exploration UI|
||A CypherQuery object||OPTIONAL Cypher query that is executed and reported to the terminal window during execution|
Recipes package together Quine config, graph logic/structure, and exploration UI customizations that are run automatically when Quine starts. A recipe file can define the graph and exploration UI configuration directly however, when a recipe is run, it infers a Quine configuration. There are a couple of operational differences that you need to keep in mind when launching Quine along with a recipe.
API calls allow naming - When you create an ingest stream or a standing query using the API, you choose the name for the object in the URL. The corresponding recipe object uses standardized names in the form of
Persistent storage - Starting the Quine application
jarwithout providing a configuration file, Quine creates a RocksDB-based persistent data store in the local directory. That persistor retains the previous graph state and is appended to each time Quine starts. Alternatively, when Quine launches a recipe, a temporary persistent data store is created in the system
tmpdirectory. Each subsequent launch of the recipe will replace the data store, discarding the graph from the previous run.
recipes are interpreted by
quine.jar using command line arguments. For help on
quine.jar command line arguments, use the following command:
❯ java -jar quine.jar -h Quine universal program Usage: quine [options] -W, --disable-web-service disable Quine web service -p, --port <value> web service port (default is 8080) -r, --recipe name, file, URL follow the specified recipe -x, --recipe-value key=value recipe parameter substitution --force-config disable recipe configuration defaults --no-delete disable deleting data file when process exits -h, --help -v, --version print Quine program version
To run a recipe, use
-r, followed by the name of the recipe (from https://quine.io/recipes), or a local file (ending in
.yaml), or a URL. recipes can expect input parameters and the parameter values must be specified using command line arguments with
A recipe may contain parameters that are used to pass information to the recipe. To use a parameter in a recipe file, a value in a recipe must start with the
$ character. The following example demonstrates a recipe with a parameter called
version: 1 title: Ingest contributor: https://github.com/landon9720 summary: Ingest input file lines as graph nodes description: Ingests each line in "$in_file" as graph node with property "line". ingestStreams: - type: FileIngest path: $in_file format: type: CypherLine query: |- MATCH (n) WHERE id(n) = idFrom($that) SET n.line = $that standingQueries: [ ] nodeAppearances: [ ] quickQueries: [ ] sampleQueries: [ ]
Running the above example (without specifying the parameter value) causes an error:
❯ java -jar quine.jar -r ingest Missing required parameter in-file; use --recipe-value in-file=
The error message indicates the command must be run with an additional command line argument that specifies the required parameter:
❯ java -jar quine.jar -r ingest --recipe-value in-file=my-file.txt
The parameter value is substituted into the recipe at runtime. Common examples for recipe parameters include file names, URLs, and host names.
When Quine is started without command line arguments, its default behavior is to start the web service on port 8080. The following options are available to change this behavior:
-W, --disable-web-service: Disables the web service
-p, --port: Specify the TCP port of the web service
Quine is configurable as described in Configuration. Normally when running a recipe, the configuration for
store is overwritten with
PersistenceAgentType.RocksDb and is configured to use a temporary file. This configuration is appropriate for most use cases of recipes and can be changed with the following command line arguments:
--force-config: Quine will use the
storethat is configured via Configuration (instead of overwriting it as described above)
--no-delete: Quine will not delete the DB file on exit and will print out that path to it
--no-deleteoptions are mutually exclusive (only one of the two is allowed at a time).
RocksDB may not function on some platforms and you need to use MapDB instead by starting Quine using parameters as follows:
java -Dquine.store.type=map-db -jar quine.jar --force-config
Complete recipes are useful as reference applications for use cases. Quine’s recipe repository highlights recipes shared by our community members. Additionally, you can view all of the shared recipes directory in the Quine Github repository.
If you make a recipe and think others might find it useful, please consider contributing it back to the repository on Github so that others can use it. They could use it on their own data, or even just use it as a starting point for customization and remixing for other goals. To share with the community, open a pull request on Github.