第 14 章索引

优质

小牛编辑

126浏览

2023-12-01

Indexing in Neo4j can be done in two different ways:

1.The database itself is a natural indexconsisting of its relationships of different types between nodes. For example a tree structure can be layered on top of the data and used for index lookups performed by a traverser.

2.Separate index engines can be used, with Apache Lucenebeing the default backend included with Neo4j.

This chapter demonstrate how to use the second type of indexing, focusing on Lucene.

14.1. Introduction

Indexing operations are part of the Neo4j index API.

Each index is tied to a unique, user-specified name (for example "first_name" or "books") and can index either nodesor relationships.

The default index implementation is provided by the neo4j-lucene-indexcomponent, which is included in the standard Neo4j download. It can also be downloaded separately from http://repo1.maven.org/maven2/org/neo4j/neo4j-lucene-index/. For Maven users, the neo4j-lucene-indexcomponent has the coordinates org.neo4j:neo4j-lucene-indexand should be used with the same version of org.neo4j:neo4j-kernel. Different versions of the index and kernel components are not compatible in the general case. Both components are included transitively by the org.neo4j:neo4j:pomartifact which makes it simple to keep the versions in sync.

For initial import of data using indexes, see 第 13.1.3 节 “批量插入数据”.

14.2. Create

An index is created if it doesn’t exist when you ask for it. Unless you give it a custom configuration, it will be created with default configuration and backend.

To set the stage for our examples, let’s create some indexes to begin with:

IndexManager index = graphDb.index();

Index<Node> actors = index.forNodes( "actors");

Index<Node> movies = index.forNodes( "movies");

RelationshipIndex roles = index.forRelationships( "roles");

This will create two node indexes and one relationship index with default configuration. See 第 14.8 节 “Relationship indexes”for more information specific to relationship indexes.

See 第 14.10 节 “Configuration and fulltext indexes”for how to create fulltextindexes.

You can also check if an index exists like this:

1 2	IndexManager index = graphDb.index(); booleanindexExists = index.existsForNodes( "actors");

14.3. Delete

Indexes can be deleted. When deleting, the entire contents of the index will be removed as well as its associated configuration. A new index can be created with the same name at a later point in time.

IndexManager index = graphDb.index();

Index<Node> actors = index.forNodes( "actors");

actors.delete();

Note that the actual deletion of the index is made during the commit of the surrounding transaction. Calls made to such an index instance after delete()has been called are invalid inside that transaction as well as outside (if the transaction is successful), but will become valid again if the transaction is rolled back.

14.4. Add

Each index supports associating any number of key-value pairs with any number of entities (nodes or relationships), where each association between entity and key-value pair is performed individually. To begin with, let’s add a few nodes to the indexes:

// Actors

Node reeves = graphDb.createNode();

reeves.setProperty( "name", "Keanu Reeves");

actors.add( reeves, "name", reeves.getProperty( "name") );

Node bellucci = graphDb.createNode();

bellucci.setProperty( "name", "Monica Bellucci");

actors.add( bellucci, "name", bellucci.getProperty( "name") );

// multiple values for a field, in this case for search only

// and not stored as a property.

actors.add( bellucci, "name", "La Bellucci");

// Movies

Node theMatrix = graphDb.createNode();

theMatrix.setProperty( "title", "The Matrix");

theMatrix.setProperty( "year", 1999);

movies.add( theMatrix, "title", theMatrix.getProperty( "title") );

movies.add( theMatrix, "year", theMatrix.getProperty( "year") );

Node theMatrixReloaded = graphDb.createNode();

theMatrixReloaded.setProperty( "title", "The Matrix Reloaded");

theMatrixReloaded.setProperty( "year", 2003);

movies.add( theMatrixReloaded, "title", theMatrixReloaded.getProperty( "title") );

movies.add( theMatrixReloaded, "year", 2003);

Node malena = graphDb.createNode();

malena.setProperty( "title", "Malèna");

malena.setProperty( "year", 2000);

movies.add( malena, "title", malena.getProperty( "title") );

movies.add( malena, "year", malena.getProperty( "year") );

Note that there can be multiple values associated with the same entity and key.

Next up, we’ll create relationships and index them as well:

// we need a relationship type

DynamicRelationshipType ACTS_IN = DynamicRelationshipType.withName( "ACTS_IN");

// create relationships

Relationship role1 = reeves.createRelationshipTo( theMatrix, ACTS_IN );

role1.setProperty( "name", "Neo");

roles.add( role1, "name", role1.getProperty( "name") );

Relationship role2 = reeves.createRelationshipTo( theMatrixReloaded, ACTS_IN );

role2.setProperty( "name", "Neo");

roles.add( role2, "name", role2.getProperty( "name") );

Relationship role3 = bellucci.createRelationshipTo( theMatrixReloaded, ACTS_IN );

role3.setProperty( "name", "Persephone");

roles.add( role3, "name", role3.getProperty( "name") );

Relationship role4 = bellucci.createRelationshipTo( malena, ACTS_IN );

role4.setProperty( "name", "Malèna Scordia");

roles.add( role4, "name", role4.getProperty( "name") );

After these operations, our example graph looks like this:

图 14.1. Movie and Actor Graph

14.5. Remove

Removingfrom an index is similar to adding, but can be done by supplying one of the following combinations of arguments:

- entity

- entity, key

- entity, key, value

// completely remove bellucci from the actors index

actors.remove( bellucci );

// remove any "name" entry of bellucci from the actors index

actors.remove( bellucci, "name");

// remove the "name" -> "La Bellucci" entry of bellucci

actors.remove( bellucci, "name", "La Bellucci");

14.6. Update

	重要
	To update an index entry, the old one must be removed and a new one added. For details on removing index entries, see 第 14.5 节 “Remove”.

Remember that a node or relationship can be associated with any number of key-value pairs in an index. This means that you can index a node or relationship with many key-value pairs that have the same key. In the case where a property value changes and you’d like to update the index, it’s not enough to just index the new value — you’ll have to remove the old value as well.

Here’s a code example that demonstrates how it’s done:

// create a node with a property

// so we have something to update later on

Node fishburn = graphDb.createNode();

fishburn.setProperty( "name", "Fishburn");

// index it

actors.add( fishburn, "name", fishburn.getProperty( "name") );

// update the index entry

// when the property value changes

actors.remove( fishburn, "name", fishburn.getProperty( "name") );

fishburn.setProperty( "name", "Laurence Fishburn");

actors.add( fishburn, "name", fishburn.getProperty( "name") );

14.7. Search

14.7.1. Get

14.7.2. Query

An index can be searched in two ways, getand query. The getmethod will return exact matches to the given key-value pair, whereas queryexposes querying capabilities directly from the backend used by the index. For example the Lucene query syntaxcan be used directly with the default indexing backend.

14.7.1. Get

This is how to search for a single exact match:

1 2	IndexHits<Node> hits = actors.get( "name", "Keanu Reeves"); Node reeves = hits.getSingle();

IndexHitsis an Iterablewith some additional useful methods. For example getSingle()returns the first and only item from the result iterator, or nullif there isn’t any hit.

Here’s how to get a single relationship by exact matching and retrieve its start and end nodes:

Relationship persephone = roles.get( "name", "Persephone").getSingle();

Node actor = persephone.getStartNode();

Node movie = persephone.getEndNode();

Finally, we can iterate over all exact matches from a relationship index:

for( Relationship role : roles.get( "name", "Neo") )

{

// this will give us Reeves twice

Node reeves = role.getStartNode();

}

重要

In case you don’t iterate through all the hits, IndexHits.close()must be called explicitly.

14.7.2. Query

有两种查询方法，其中使用键-值签名的值与给定键只表示值的查询的位置。另一种方法是更通用，并支持查询在同一查询中的多个键-值对。

下面是一个示例使用键查询选项：

for( Node actor : actors.query( "name", "*e*") )

{

// This will return Reeves and Bellucci

}

In the following example the query uses multiple keys:

for( Node movie : movies.query( "title:*Matrix* AND year:1999") )

{

// This will return "The Matrix" from 1999 only.

}

注意

Beginning a wildcard search with "*" or "?" is discouraged by Lucene, but will nevertheless work.

	小心
	You can’t have any whitespacein the search term with this syntax. See 第 14.11.3 节 “Querying with Lucene Query objects”for how to do that.

14.8. Relationship indexes

An index for relationships is just like an index for nodes, extended by providing support to constrain a search to relationships with a specific start and/or end nodes These extra methods reside in the RelationshipIndexinterface which extends Index<Relationship>.

Example of querying a relationship index:

// find relationships filtering on start node

// using exact matches

IndexHits<Relationship> reevesAsNeoHits;

reevesAsNeoHits = roles.get( "name", "Neo", reeves, null);

Relationship reevesAsNeo = reevesAsNeoHits.iterator().next();

reevesAsNeoHits.close();

// find relationships filtering on end node

// using a query

IndexHits<Relationship> matrixNeoHits;

matrixNeoHits = roles.query( "name", "*eo", null, theMatrix );

Relationship matrixNeo = matrixNeoHits.iterator().next();

matrixNeoHits.close();

And here’s an example for the special case of searching for a specific relationship type:

// find relationships filtering on end node

// using a relationship type.

// this is how to add it to the index:

roles.add( reevesAsNeo, "type", reevesAsNeo.getType().name() );

// Note that to use a compound query, we can't combine committed

// and uncommitted index entries, so we'll commit before querying:

tx.success();

tx.finish();

// and now we can search for it:

IndexHits<Relationship> typeHits;

typeHits = roles.query( "type:ACTS_IN AND name:Neo", null, theMatrix );

Relationship typeNeo = typeHits.iterator().next();

typeHits.close();

Such an index can be useful if your domain has nodes with a very large number of relationships between them, since it reduces the search time for a relationship between two nodes. A good example where this approach pays dividends is in time series data, where we have readings represented as a relationship per occurrence.

14.9. Scores

The IndexHitsinterface exposes scoringso that the index can communicate scores for the hits. Note that the result is not sorted by the score unless you explicitly specify that. See 第 14.11.2 节 “Sorting”for how to sort by score.

IndexHits<Node> hits = movies.query( "title", "The*");

for( Node movie : hits )

{

System.out.println( movie.getProperty( "title") + " "+ hits.currentScore() );

}

14.10. Configuration and fulltext indexes

At the time of creation extra configuration can be specified to control the behavior of the index and which backend to use. For example to create a Lucene fulltext index:

IndexManager index = graphDb.index();

Index<Node> fulltextMovies = index.forNodes( "movies-fulltext",

MapUtil.stringMap( IndexManager.PROVIDER, "lucene", "type", "fulltext") );

fulltextMovies.add( theMatrix, "title", "The Matrix");

fulltextMovies.add( theMatrixReloaded, "title", "The Matrix Reloaded");

// search in the fulltext index

Node found = fulltextMovies.query( "title", "reloAdEd").getSingle();

Here’s an example of how to create an exact index which is case-insensitive:

Index<Node> index = graphDb.index().forNodes( "exact-case-insensitive",

stringMap( "type", "exact", "to_lower_case", "true") );

Node node = graphDb.createNode();

index.add( node, "name", "Thomas Anderson");

assertContains( index.query( "name", "\"Thomas Anderson\""), node );

assertContains( index.query( "name", "\"thoMas ANDerson\""), node );

提示

In order to search for tokenized words, the querymethod has to be used. The getmethod will only match the full string value, not the tokens.

The configuration of the index is persisted once the index has been created. The providerconfiguration key is interpreted by Neo4j, but any other configuration is passed onto the backend index (e.g. Lucene) to interpret.

表 14.1. Lucene indexing configuration parameters

Parameter	Possible values	Effect
type	exact, fulltext	exactis the default and uses a Lucene keyword analyzer. fulltextuses a white-space tokenizer in its analyzer.
to_lower_case	true, false	This parameter goes together with type: fulltextand converts values to lower case during both additions and querying, making the index case insensitive. Defaults to true.
analyzer	the full class name of an Analyzer	Overrides the typeso that a custom analyzer can be used. Note: to_lower_casestill affects lowercasing of string queries. If the custom analyzer uppercases the indexed tokens, string queries will not match as expected.

14.11. Extra features for Lucene indexes

14.11.1. Numeric ranges

14.11.2. Sorting

14.11.3. Querying with Lucene Query objects

14.11.4. Compound queries

14.11.5. Default operator

14.11.6. Caching

14.11.1. Numeric ranges

Lucene supports smart indexing of numbers, querying for ranges and sorting such results, and so does its backend for Neo4j. To mark a value so that it is indexed as a numeric value, we can make use of the ValueContextclass, like this:

movies.add( theMatrix, "year-numeric", newValueContext( 1999).indexNumeric() );

movies.add( theMatrixReloaded, "year-numeric", newValueContext( 2003).indexNumeric() );

movies.add( malena, "year-numeric", newValueContext( 2000).indexNumeric() );

intfrom = 1997;

intto = 1999;

hits = movies.query( QueryContext.numericRange( "year-numeric", from, to ) );

注意

The same type must be used for indexing and querying. That is, you can’t index a value as a Long and then query the index using an Integer.

By giving nullas from/to argument, an open ended query is created. In the following example we are doing that, and have added sorting to the query as well:

hits = movies.query(

QueryContext.numericRange( "year-numeric", from, null)

.sortNumeric( "year-numeric", false) );

From/to in the ranges defaults to be inclusive, but you can change this behavior by using two extra parameters:

movies.add( theMatrix, "score", newValueContext( 8.7).indexNumeric() );

movies.add( theMatrixReloaded, "score", newValueContext( 7.1).indexNumeric() );

movies.add( malena, "score", newValueContext( 7.4).indexNumeric() );

// include 8.0, exclude 9.0

hits = movies.query( QueryContext.numericRange( "score", 8.0, 9.0, true, false) );

14.11.2. Sorting

Lucene performs sorting very well, and that is also exposed in the index backend, through the QueryContextclass:

hits = movies.query( "title", newQueryContext( "*").sort( "title") );

for( Node hit : hits )

{

// all movies with a title in the index, ordered by title

}

// or

hits = movies.query( newQueryContext( "title:*").sort( "year", "title") );

for( Node hit : hits )

{

// all movies with a title in the index, ordered by year, then title

}

We sort the results by relevance (score) like this:

hits = movies.query( "title", newQueryContext( "The*").sortByScore() );

for( Node movie : hits )

{

// hits sorted by relevance (score)

}

14.11.3. Querying with Lucene Query objects

Instead of passing in Lucene query syntax queries, you can instantiate such queries programmatically and pass in as argument, for example:

1 2	// a TermQuery will give exact matches Node actor = actors.query( newTermQuery( newTerm( "name", "Keanu Reeves") ) ).getSingle();

Note that the TermQueryis basically the same thing as using the getmethod on the index.

This is how to perform wildcardsearches using Lucene Query Objects:

hits = movies.query( newWildcardQuery( newTerm( "title", "The Matrix*") ) );

for( Node movie : hits )

{

System.out.println( movie.getProperty( "title") );

}

Note that this allows for whitespace in the search string.

14.11.4. Compound queries

Lucene supports querying for multiple terms in the same query, like so:

	1	hits = movies.query( "title:Matrix AND year:1999");
		小心
		Compound queries can’t search across committed index entries and those who haven’t got committed yet at the same time.

14.11.5. Default operator

The default operator (that is whether ANDor ORis used in between different terms) in a query is OR. Changing that behavior is also done via the QueryContextclass:

QueryContext query = newQueryContext( "title:*Matrix* year:1999")

.defaultOperator( Operator.AND );

hits = movies.query( query );

14.11.6. Caching

If your index lookups becomes a performance bottle neck, caching can be enabled for certain keys in certain indexes (key locations) to speed up get requests. The caching is implemented with an LRUcache so that only the most recently accessed results are cached (with "results" meaning a query result of a get request, not a single entity). You can control the size of the cache (the maximum number of results) per index key.

Index<Node> index = graphDb.index().forNodes( "actors");

( (LuceneIndex<Node>) index ).setCacheCapacity( "name", 300000);

小心

This setting is not persisted after shutting down the database. This means: set this value after each startup of the database if you want to keep it.

14.12. Automatic Indexing

14.12.1. Configuration

14.12.2. Search

14.12.3. Runtime Configuration

14.12.4. Updating the Automatic Index

Neo4j provides a single index for nodes and one for relationships in each database that automatically follow property values as they are added, deleted and changed on database primitives. This functionality is called auto indexingand is controlled both from the database configuration Map and through its own API.

14.12.1. Configuration

By default Auto Indexing is off for both Nodes and Relationships. To enable it on database startup set the configuration options Config.NODE_AUTO_INDEXINGand Config.RELATIONSHIP_AUTO_INDEXINGto the string "true".

If you just enable auto indexing as above, then still noproperty will be auto indexed. To define which property names you want the auto indexer to monitor as a configuration parameter, set the Config.{NODE,RELATIONSHIP}_KEYS_INDEXABLEoption to a String that is a comma separated concatenation of the property names you want auto indexed.

* Creating the configuration, adding nodeProp1 and nodeProp2 as

* auto indexed properties for Nodes and relProp1 and relProp2 as

* auto indexed properties for Relationships. Only those will be

* indexed. We also have to enable auto indexing for both these

* primitives explicitly.

GraphDatabaseService graphDb = newGraphDatabaseFactory().

newEmbeddedDatabaseBuilder( storeDirectory ).

setConfig( GraphDatabaseSettings.node_keys_indexable, "nodeProp1,nodeProp2").

setConfig( GraphDatabaseSettings.relationship_keys_indexable, "relProp1,relProp2").

setConfig( GraphDatabaseSettings.node_auto_indexing, "true").

setConfig( GraphDatabaseSettings.relationship_auto_indexing, "true").

newGraphDatabase();

Transaction tx = graphDb.beginTx();

Node node1 = null, node2 = null;

Relationship rel = null;

try

{

// Create the primitives

node1 = graphDb.createNode();

node2 = graphDb.createNode();

rel = node1.createRelationshipTo( node2,

DynamicRelationshipType.withName( "DYNAMIC") );

// Add indexable and non-indexable properties

node1.setProperty( "nodeProp1", "nodeProp1Value");

node2.setProperty( "nodeProp2", "nodeProp2Value");

node1.setProperty( "nonIndexed", "nodeProp2NonIndexedValue");

rel.setProperty( "relProp1", "relProp1Value");

rel.setProperty( "relPropNonIndexed", "relPropValueNonIndexed");

// Make things persistent

tx.success();

}

catch( Exception e )

{

tx.failure();

}

finally

{

tx.finish();

}

14.12.2. Search

The usefulness of the auto indexing functionality comes of course from the ability to actually query the index and retrieve results. To that end, you can acquire a ReadableIndexobject from the AutoIndexerthat exposes all the query and get methods of a full Indexwith exactly the same functionality. Continuing from the previous example, accessing the index is done like this:

// Get the Node auto index

ReadableIndex<Node> autoNodeIndex = graphDb.index()

.getNodeAutoIndexer()

.getAutoIndex();

// node1 and node2 both had auto indexed properties, get them

assertEquals( node1,

autoNodeIndex.get( "nodeProp1", "nodeProp1Value").getSingle() );

assertEquals( node2,

autoNodeIndex.get( "nodeProp2", "nodeProp2Value").getSingle() );

// node2 also had a property that should be ignored.

assertFalse( autoNodeIndex.get( "nonIndexed",

"nodeProp2NonIndexedValue").hasNext() );

// Get the relationship auto index

ReadableIndex<Relationship> autoRelIndex = graphDb.index()

.getRelationshipAutoIndexer()

.getAutoIndex();

// One property was set for auto indexing

assertEquals( rel,

autoRelIndex.get( "relProp1", "relProp1Value").getSingle() );

// The rest should be ignored

assertFalse( autoRelIndex.get( "relPropNonIndexed",

"relPropValueNonIndexed").hasNext() );

14.12.3. Runtime Configuration

The same options that are available during database creation via the configuration can also be set during runtime via the AutoIndexerAPI.

Gaining access to the AutoIndexerAPI and adding two Nodeand one Relationshipproperties to auto index is done like so:

// Start without any configuration

GraphDatabaseService graphDb = newGraphDatabaseFactory().

newEmbeddedDatabase( storeDirectory );

// Get the Node AutoIndexer, set nodeProp1 and nodeProp2 as auto

// indexed.

AutoIndexer<Node> nodeAutoIndexer = graphDb.index()

.getNodeAutoIndexer();

nodeAutoIndexer.startAutoIndexingProperty( "nodeProp1");

nodeAutoIndexer.startAutoIndexingProperty( "nodeProp2");

// Get the Relationship AutoIndexer

AutoIndexer<Relationship> relAutoIndexer = graphDb.index()

.getRelationshipAutoIndexer();

relAutoIndexer.startAutoIndexingProperty( "relProp1");

// None of the AutoIndexers are enabled so far. Do that now

nodeAutoIndexer.setEnabled( true);

relAutoIndexer.setEnabled( true);

Parameters to the AutoIndexers passed through the Configuration and settings made through the API are cumulative. So you can set some beforehand known settings, do runtime checks to augment the initial configuration and then enable the desired auto indexers - the final configuration is the same regardless of the method used to reach it.

14.12.4. Updating the Automatic Index

Updates to the auto indexed properties happen of course automatically as you update them. Removal of properties from the auto index happens for two reasons. One is that you actually removed the property. The other is that you stopped autoindexing on a property. When the latter happens, any primitive you touch and it has that property, it is removed from the auto index, regardless of any operations on the property. When you start or stop auto indexing on a property, no auto update operation happens currently. If you need to change the set of auto indexed properties and have them re-indexed, you currently have to do this by hand. An example will illustrate the above better:

* Creating the configuration

GraphDatabaseService graphDb = new GraphDatabaseFactory().

newEmbeddedDatabaseBuilder( storeDirectory ).

setConfig( GraphDatabaseSettings.node_keys_indexable, "nodeProp1,nodeProp2" ).

setConfig( GraphDatabaseSettings.node_auto_indexing, "true" ).

newGraphDatabase();

Transaction tx = graphDb.beginTx();

Node node1 = null, node2 = null, node3 = null, node4 = null;

try

{

// Create the primitives

node1 = graphDb.createNode();

node2 = graphDb.createNode();

node3 = graphDb.createNode();

node4 = graphDb.createNode();

// Add indexable and non-indexable properties

node1.setProperty( "nodeProp1", "nodeProp1Value" );

node2.setProperty( "nodeProp2", "nodeProp2Value" );

node3.setProperty( "nodeProp1", "nodeProp3Value" );

node4.setProperty( "nodeProp2", "nodeProp4Value" );

// Make things persistent

tx.success();

}

catch ( Exception e )

{

tx.failure();

}

finally

{

tx.finish();

}

* Here both nodes are indexed. To demonstrate removal, we stop

* autoindexing nodeProp1.

AutoIndexer<Node> nodeAutoIndexer = graphDb.index().getNodeAutoIndexer();

nodeAutoIndexer.stopAutoIndexingProperty( "nodeProp1" );

tx = graphDb.beginTx();

try

{

* nodeProp1 is no longer auto indexed. It will be

* removed regardless. Note that node3 will remain.

node1.setProperty( "nodeProp1", "nodeProp1Value2" );

* node2 will be auto updated

node2.setProperty( "nodeProp2", "nodeProp2Value2" );

* remove node4 property nodeProp2 from index.

node4.removeProperty( "nodeProp2" );

// Make things persistent

tx.success();

}

catch ( Exception e )

{

tx.failure();

}

finally

{

tx.finish();

}

// Verify

ReadableIndex<Node> nodeAutoIndex = nodeAutoIndexer.getAutoIndex();

// node1 is completely gone

assertFalse( nodeAutoIndex.get( "nodeProp1", "nodeProp1Value" ).hasNext() );

assertFalse( nodeAutoIndex.get( "nodeProp1", "nodeProp1Value2" ).hasNext() );

// node2 is updated

assertFalse( nodeAutoIndex.get( "nodeProp2", "nodeProp2Value" ).hasNext() );

assertEquals( node2,

nodeAutoIndex.get( "nodeProp2", "nodeProp2Value2" ).getSingle() );

* node3 is still there, despite its nodeProp1 property not being monitored

* any more because it was not touched, in contrast with node1.

assertEquals( node3,

nodeAutoIndex.get( "nodeProp1", "nodeProp3Value").getSingle() );

// Finally, node4 is removed because the property was removed.

assertFalse( nodeAutoIndex.get( "nodeProp2", "nodeProp4Value").hasNext() );

小心

If you start the database with auto indexing enabled but different auto indexed properties than the last run, then already auto-indexed entities will be deleted as you work with them. Make sure that the monitored set is what you want before enabling the functionality.

第 14 章 索引

第 14 章索引