© 2018-2024 The original authors.
Note
|
Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically. |
- Preface
- Reference documentation
- Functionality
- Installation & Usage
- Aerospike repositories
- Reactive Aerospike repositories
- Projections with Aerospike
- Query Methods
- Simple Property Repository Queries
- Collection Repository Queries
- Map Repository Queries
- POJO Repository Queries
- Id Repository Queries
- Combined Query Methods
- Query Modification
- Aerospike Object Mapping
- Aerospike Custom Converters
- Aerospike Template
- Secondary indexes
- Indexed Annotation
- Caching
- Configuration
- Appendix
Preface
The Spring Data Aerospike project applies core Spring concepts and provides interface for using Aerospike key-value style data store. We provide a "repository" and a "template" as high-level abstractions for storing and querying data. You will notice similarities to the JDBC support in the Spring Framework.
This chapter provides some basic introduction to Spring and Aerospike, it explains Aerospike concepts and syntax. The rest of the documentation refers to Spring Data Aerospike features and assumes the user is familiar with Aerospike as well as Spring concepts.
Knowing Spring
Spring Data uses Spring framework’s core functionality, such as the IoC container, type conversion system, DAO exception hierarchy etc. While it is not important to know the Spring APIs, understanding the concepts behind them is. At a minimum, the idea behind IoC should be familiar regardless of IoC container you choose to use.
To learn more about Spring, you can refer to the comprehensive documentation that explains in detail the Spring Framework. There are a lot of articles, blog entries and books on the matter - take a look at the Spring framework documentation reference for more information.
Knowing NoSQL and Aerospike
NoSQL stores have taken the storage world by storm. It is a vast domain with a plethora of solutions, terms and patterns (to make things worthwhile even the term itself has multiple meanings). While some principles are common, it is crucial that the user is familiar to some degree with Aerospike key-value store operations that supply the mechanism for associating keys with a set of named values, similar to a row in standard RDBMS terminology. The data layer in Aerospike Database is optimized to store data in solid state drives, RAM, or traditional rotational media. The database indices are stored in RAM for quick availability, and data writes are optimized through large block writes to reduce latency. The software also employs two sub-programs that are codenamed Defragmenter and Evictor. Defragmenter removes data blocks that have been deleted, and Evictor frees RAM space by removing references to expired records.
The jumping off ground for learning about Aerospike is www.aerospike.com. Here is a list of other useful resources:
-
The technical documentation introduces Aerospike and contains links to getting started guides, reference documentation and tutorials.
-
The java client documentation provides a convenient way to interact with an Aerospike instance in combination with the online Getting started
Requirements
Spring Data Aerospike binaries require JDK level 17.0 and above.
In terms of server, it is required to use at least Aerospike server version 6.1 (recommended to use the latest version when possible).
Additional Help Resources
Learning a new framework is not always straightforward. In this section, we try to provide what we think is an easy-to-follow guide for starting with Spring Data Aerospike module. However, if you encounter issues, or you are just looking for advice, feel free to use one of the links below:
Support
There are a few support options available:
Questions & Answers
Developers post questions and answers on Stack Overflow. The two key tags to search for related answers to this project are:
Following Development
If you encounter a bug or want to suggest an improvement, please create an issue on GitHub.
Reference documentation
Functionality
Spring Data Aerospike project aims to provide a familiar and consistent Spring-based programming model providing integration with the Aerospike database.
Spring Data Aerospike supports a wide range of features summarized below:
-
Supporting Repository interfaces (out-of-the-box CRUD operations and query implementations, for more information see Aerospike Repositories)
-
AerospikeTemplate for lower-level access to common Aerospike operations and fine-tuning (for more information see AerospikeTemplate)
-
Feature Rich Object Mapping integrated with Spring’s Conversion Service
-
Translating exceptions into Spring’s Data Access Exception hierarchy
-
Annotation-based metadata mapping
-
Ability to directly utilize Aerospike Java client functionality
Installation & Usage
Getting Started
First, you need a running Aerospike server to connect to.
To use Spring Data Aerospike you can either set up Spring Boot or Spring application. Basic setup of Spring Boot application is described here: https://projects.spring.io/spring-boot.
In case you do not want to use Spring Boot, the best way to manage Spring dependencies is to declare spring-framework-bom
of the needed version in the dependencyManagement
section of your pom.xml
:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-framework-bom</artifactId>
<version>${spring-data-aerospike.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Note
|
To create a Spring project in STS (Spring Tool Suite) go to File → New → Spring Template Project → Simple Spring Utility Project → press "Yes" when prompted. Then enter a project and a package name such as org.spring.aerospike.example .
|
Adding Dependency
The first step is to add Spring Data Aerospike to your build process. It is recommended to use the latest version which can be found on the GitHub Releases page.
Adding Spring Data Aerospike dependency in Maven:
<dependency>
<groupId>com.aerospike</groupId>
<artifactId>spring-data-aerospike</artifactId>
<version>${spring-data-aerospike.version}</version>
</dependency>
Adding Spring Data Aerospike dependency in Gradle:
implementation group: 'com.aerospike', name: 'spring-data-aerospike', version: '${spring-data-aerospike.version}'
Connecting to Aerospike DB
There are two ways of configuring a basic connection to Aerospike DB.
-
Overriding
getHosts()
andnameSpace()
methods of theAbstractAerospikeDataConfiguration
class:
@Configuration
@EnableAerospikeRepositories(basePackageClasses = { PersonRepository.class})
public class AerospikeConfiguration extends AbstractAerospikeDataConfiguration {
@Override
protected Collection<Host> getHosts() {
return Collections.singleton(new Host("localhost", 3000));
}
@Override
protected String nameSpace() {
return "test";
}
}
-
Using
application.properties
:
Basic configuration in this case requires enabling repositories and then setting hosts
and namespace
in the application.properties
file.
@Configuration
@EnableAerospikeRepositories(basePackageClasses = { PersonRepository.class})
public class AerospikeConfiguration extends AbstractAerospikeDataConfiguration {
}
In application.properties
:
# application.properties
spring-data-aerospike.hosts=localhost:3000
spring-data-aerospike.namespace=test
Note
|
Return values of getHosts() and nameSpace() methods of the AbstractAerospikeDataConfiguration class
have precedence over hosts and namespace parameters set via application.properties.
|
For more detailed information see Configuration.
Creating Functionality
The base functionality is provided by AerospikeRepository
interface.
It typically takes 2 parameters:
-
The type managed by a class (it is typically entity class) to be stored in the database.
-
The type of ID.
Application code typically extends this interface for each of the types to be managed, and methods can be added to the interface to determine how the application can access the data. For example, consider a class Person
with a simple structure:
@AllArgsConstructor
@NoArgsConstructor
@Data
@Document
public class Person {
@Id
private long id;
private String firstName;
private String lastName;
@Field("dob")
private Date dateOfBirth;
}
Note that this example uses the Project Lombok annotations to remove the need for explicit constructors and getters and setters. Normal POJOs which define these on their own can ignore the @AllArgsConstructor
, @NoArgsConstructor
and @Data
annotations. The @Document
annotation tells Spring Data Aerospike that this is a domain object to be persisted in the database, and @Id
identifies the primary key of this class. The @Field
annotation is used to create a shorter name for the bin in the Aerospike database (dateOfBirth
will be stored in a bin called dob
in this example).
For the Person
object to be persisted to Aerospike, you must create an interface with the desired methods for retrieving data. For example:
public interface PersonRepository extends AerospikeRepository<Person, Long> {
List<Person> findByLastName(String lastName);
}
This defines a repository that can write Person
entities and also query them by last name. The AerospikeRepository
extends both PagingAndSortingRepository
and CrudRepository
, so methods like count()
, findById()
, save()
and delete()
are there by default. Those who need reactive flow can use ReactiveAerospikeRepository
instead.
Note
|
Repository is just an interface and not an actual class. In the background, when your context gets initialized, actual implementations for your repository descriptions get created, and you can access them through regular beans. This means you will omit lots of boilerplate code while still exposing full CRUD semantics to your service layer and application. |
Example repository is ready for use. A sample Spring Controller which uses this repository could be the following:
@RestController
public class ApplicationController {
@Autowired
private PersonRepository personRepsitory;
@GetMapping("/seed")
public int seedData() {
Person person = new Person(1, "Bob", "Jones", new GregorianCalendar(1971, 12, 19).getTime());
personRepsitory.save(person);
return 1;
}
@GetMapping("/findByLastName/{lastName}")
public List<Person> findByLastName(@PathVariable(name = "lastName", required=true) String lastName) {
return personRepsitory.findByLastName(lastName);
}
}
Invoking the seed
method above gives you a record in the Aerospike database which looks like:
aql> select * from test.Person where pk = "1"
+-----+-----------+----------+-------------+-------------------------------------+
| PK | firstName | lastName | dob | @_class |
+-----+-----------+----------+-------------+-------------------------------------+
| "1" | "Bob" | "Jones" | 64652400000 | "com.aerospike.sample.model.Person" |
+-----+-----------+----------+-------------+-------------------------------------+
1 row in set (0.001 secs)
Note
|
The fully qualified path of the class is listed in each record. This is needed to instantiate the class correctly, especially in cases when the compile-time type and runtime type of the object differ. For example, where a field is declared as a super class but the instantiated class is a subclass. |
Note
|
By default, the type of the field annotated with @id is turned into a String to be stored in Aerospike database. If the original type cannot be persisted (see keepOriginalKeyTypes for details), it must be convertible to String and will be stored in the database as such, then converted back to the original type when the object is read. This is transparent to the application but needs to be considered if using external tools like AQL to view the data.
|
Aerospike repositories
Introduction
One of the main goals of the Spring Data is to significantly reduce the amount of boilerplate code required to implement data access layers for various persistence stores.
One of the core interfaces of Spring Data is Repository
.
This interface acts primarily to capture the types to work with and to help user to discover interfaces that extend Repository.
In other words, it allows user to have basic and complicated queries without writing the implementation. This builds on the Core Spring Data Repository Support, so make sure you’ve got a sound understanding of this concept.
Usage
To access entities stored in Aerospike you can leverage repository support that eases implementing those quite significantly. To do so, simply create an interface for your repository:
public class Person {
@Id
private String id;
private String name;
private int age;
public Person(String id, String name, int age) {
this.id = id;
this.name = name;
this.age = age;
}
// … getters and setters omitted
}
We have a quite simple domain object here. The default serialization mechanism used in AerospikeTemplate
(which is backing the repository support) regards properties named "id" as document id. Currently we support String
and long
as id-types.
public interface PersonRepository extends AerospikeRepository<Person, String> {
List<Person> findByName(String name);
List<Person> findByNameStartsWith(String prefix);
}
Right now this interface simply serves typing purposes, but we will add additional methods to it later. In your Spring configuration simply add
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:aerospike="http://www.springframework.org/schema/data/aerospike"
xsi:schemaLocation="http://www.springframework.org/schema/beans
https://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/data/aerospike
https://www.springframework.org/schema/data/aerospike/spring-aerospike-1.0.xsd">
<aerospike:aerospike id="aerospike" />
<bean id="aerospikeTemplate" class="rg.springframework.data.aerospike.core.AerospikeTemplate">
</bean>
<aerospike:repositories base-package="org.springframework.data.aerospike.example.data" />
</beans>
This namespace element will cause the base packages to be scanned for interfaces extending AerospikeRepository
and create Spring beans for each of them found. By default, the repositories will get an AerospikeTemplate
Spring bean wired that is called aerospikeTemplate
.
If you’d rather like to go with JavaConfig use the @EnableAerospikeRepositories
annotation. The annotation carries the very same attributes like the namespace element. If no base package is configured the infrastructure will scan the package of the annotated configuration class.
@Configuration
@EnableAerospikeRepositories(basePackages = "org.springframework.data.aerospike.example")
public class TestRepositoryConfig {
public @Bean(destroyMethod = "close") AerospikeClient aerospikeClient() {
ClientPolicy policy = new ClientPolicy();
policy.failIfNotConnected = true;
return new AerospikeClient(policy, "52.23.205.208", 3000);
}
public @Bean AerospikeTemplate aerospikeTemplate() {
return new AerospikeTemplate(aerospikeClient(), "test");
}
}
As our domain repository extends PagingAndSortingRepository
it provides you with CRUD operations as well as methods for paginated and sorted access to the entities. Working with the repository instance is just a matter of dependency injecting it into a client. So accessing the second page of `Person`s at a page size of 10 would simply look something like this:
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class PersonRepositoryTests {
@Autowired PersonRepository repository;
@Test
public void readsFirstPageCorrectly() {
Page<Person> persons = repository.findAll(new PageRequest(0, 10));
assertThat(persons.isFirstPage(), is(true));
}
}
The sample creates an application context with Spring’s unit test support which will perform annotation-based dependency injection into test cases. Inside the test method, we simply use the repository to query the datastore. We hand the repository a PageRequest
instance that requests the first page of persons at a page size of 10.
Query methods
Most of the data access operations you usually trigger on a repository result in a query being executed against the Aerospike databases. Defining such a query is just a matter of declaring a method on the repository interface
public interface PersonRepository extends PagingAndSortingRepository<Person, String> {
List<Person> findByName(String name); (1)
Page<Person> findByName(String name, Pageable pageable); (2)
List<Person> findByNameStartsWith(String prefix); (3)
}
-
The method shows a query for all people with the given name. The query will be derived by parsing the method name for constraints that can be concatenated with
And
andOr
. -
Applies pagination to a query. Just equip your method signature with a
Pageable
parameter and let the method return aPage
instance, and it will automatically page the query accordingly (i.e. return the required part of results). -
Uses query-based partial name search.
Here’s a delete insert and query example
@ContextConfiguration(classes = TestRepositoryConfig.class)
public class RepositoryExample {
@Autowired
protected PersonRepository repository;
@Autowired
AerospikeOperations aerospikeOperations;
@Autowired
AerospikeClient client;
public RepositoryExample(ApplicationContext ctx) {
aerospikeOperations = ctx.getBean(AerospikeTemplate.class);
repository = (PersonRepository) ctx.getBean("personRepository");
client = ctx.getBean(AerospikeClient.class);
}
protected void setUp() {
repository.deleteAll();
Person dave = new Person("Dave-01", "Matthews", 42);
Person donny = new Person("Dave-02", "Macintire", 39);
Person oliver = new Person("Oliver-01", "Matthews", 4);
Person carter = new Person("Carter-01", "Beauford", 49);
Person boyd = new Person("Boyd-01", "Tinsley", 45);
Person stefan = new Person("Stefan-01", "Lessard", 34);
Person leroi = new Person("Leroi-01", "Moore", 41);
Person leroi2 = new Person("Leroi-02", "Moore", 25);
Person alicia = new Person("Alicia-01", "Keys", 30);
repository.createIndex(Person.class, "person_name_index", "name",
IndexType.STRING);
List<Person> all = (List<Person>) repository.save(Arrays.asList(oliver,
dave, donny, carter, boyd, stefan, leroi, leroi2, alicia));
}
protected void cleanUp() {
repository.deleteAll();
}
protected void executeRepositoryCall() {
List<Person> result = repository.findByName("Beauford");
System.out.println("Results for exact match of 'Beauford'");
for (Person person : result) {
System.out.println(person.toString());
}
System.out.println("Results for name starting with letter 'M'");
List<Person> resultPartial = repository.findByNameStartsWith("M");
for (Person person : resultPartial) {
System.out.println(person.toString());
}
}
public static void main(String[] args) {
ApplicationContext ctx = new AnnotationConfigApplicationContext(
TestRepositoryConfig.class);
RepositoryExample repositoryExample = new RepositoryExample(ctx);
repositoryExample.setUp();
repositoryExample.executeRepositoryCall();
repositoryExample.cleanUp();
}
}
Reactive Aerospike repositories
Introduction
This chapter will point out the specialties for reactive repository support for Aerospike. This builds on the core repository support explained in repositories. So make sure you’ve got a sound understanding of the basic concepts explained there.
Reactive Composition Libraries
The reactive space offers various reactive composition libraries. The most common library is Project Reactor.
Spring Data Aerospike is built on top of the Aerospike Reactor Java Client Library, to provide maximal interoperability by relying on the Reactive Streams initiative. Static APIs, such as ReactiveAerospikeOperations
, are provided by using Project Reactor’s Flux
and Mono
types. Project Reactor offers various adapters to convert reactive wrapper types (Flux
to Observable
and vice versa).
Spring Data’s Repository abstraction is a dynamic API, mostly defined by you and your requirements as you declare query methods. Reactive Aerospike repositories can be implemented by using Project Reactor wrapper types by extending from the following library-specific repository interface:
-
ReactiveAerospikeRepository
Usage
To access domain entities stored in an Aerospike you can use our sophisticated repository support that eases implementing those quite significantly. To do so, create an interface similar to your repository. Before you can do that, though, you need an entity, such as the entity defined in the following example:
Person
entitypublic class Person {
@Id
private String id;
private String firstname;
private String lastname;
private Address address;
// … getters and setters omitted
}
We have a quite simple domain object here. The default serialization mechanism used in ReactiveAerospikeTemplate
(which is backing the repository support) regards properties named id as document ID. Currently, we support String
and long
as id-types. The following example shows how to create an interface that defines queries against the Person
object from the preceding example:
public interface ReactivePersonRepository extends ReactiveAerospikeRepository<Person, String> {
Flux<Person> findByFirstname(String firstname);
}
Right now this interface simply serves typing purposes but we will add additional methods to it later.
For Java configuration, use the @EnableReactiveAerospikeRepositories
annotation. The annotation carries the base
packages attribute. These base packages are to be scanned for interfaces extending ReactiveAerospikeRepository
and create Spring beans for each of them found. If no base package is configured, the infrastructure scans the package
of the annotated configuration class.
The following listing shows how to use Java configuration for a repository:
@Configuration
@EnableReactiveAerospikeRepositories(basePackages = "org.springframework.data.aerospike.example")
public class TestRepositoryConfig extends AbstractReactiveAerospikeDataConfiguration {
@Override
protected Collection<Host> getHosts() {
return Collections.singleton(new Host("52.23.205.208", 3000));
}
@Override
protected String nameSpace() {
return "test";
}
@Override
protected EventLoops eventLoops() {
return new NioEventLoops();
}
}
As our domain repository extends ReactiveAerospikeRepository
it provides you with CRUD operations. Working with the repository instance is a matter of dependency injecting it into a client, as the following example shows:
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class PersonRepositoryTests {
@Autowired
ReactivePersonRepository repository;
@Test
public void findByFirstnameCorrectly() {
Flux<Person> persons = repository.findByFirstname("TestFirstName");
}
}
The sample creates an application context with Spring’s unit test support which will perform annotation-based dependency injection into test cases. Inside the test method, we simply use the repository to query the datastore.
Query methods
Most of the data access operations you usually trigger on a repository result in a query being executed against the Aerospike databases. Defining such a query is just a matter of declaring a method on the repository interface
public interface ReactivePersonRepository extends ReactiveAerospikeRepository<Person, String> {
Flux<Person> findByFirstname(String firstname); (1)
Flux<Person> findByFirstname(Publisher<String> firstname); (2)
Mono<Person> findByFirstnameAndLastname(String firstname, String lastname); (3)
Mono<Person> findFirstByLastname(String lastname); (4)
Flux<Person> findByFirstnameStartsWith(String prefix); (5)
}
-
The method shows a query for all people with the given
firstname
. The query is derived by parsing the method name for constraints that can be concatenated withAnd
andOr
. Thus, the method name results in a query expression of{"firstname" : firstname}
. -
The method shows a query for all people with the given
firstname
once thefirstname
is emitted by the givenPublisher
. -
Find a single entity for the given criteria. It completes with
IncorrectResultSizeDataAccessException
on non-unique results. -
Unless <3>, the first entity is always emitted even if the query yields more result documents.
-
The method shows a query for all people with the firstname starts from
prefix
Examples
Here’s a delete, insert and query example
@ContextConfiguration(classes = TestRepositoryConfig.class)
public class ReactiveRepositoryExample {
@Autowired
protected ReactivePersonRepository repository;
@Autowired
ReactiveAerospikeOperations aerospikeOperations;
@Autowired
IAerospikeReactorClient client;
public RepositoryExample(ApplicationContext ctx) {
aerospikeOperations = ctx.getBean(ReactiveAerospikeTemplate.class);
repository = (ReactivePersonRepository) ctx.getBean("reactivePersonRepository");
client = ctx.getBean(IAerospikeReactorClient.class);
}
protected void setUp() {
// Insert new Person items into repository
Person dave = new Person("Dave-01", "Matthews", 42);
Person donny = new Person("Dave-02", "Macintire", 39);
Person oliver = new Person("Oliver-01", "Matthews", 4);
Person carter = new Person("Carter-01", "Beauford", 49);
List<Person> all = saveAll(Arrays.asList(dave, donny, oliver, carter))
.collectList().block();
}
protected void cleanUp() {
// Delete all Person items from repository
repository.findAll().flatMap(a -> repository.delete(a)).blockLast();
}
protected void executeRepositoryCall() {
System.out.println("Results for first name exact match of 'Dave-02'");
repository.findByFirstname("Dave-02")
.doOnNext(person -> System.out.println(person.toString())).blockLast();
System.out.println("Results for first name starting with letter 'D'");
repository.findByFirstnameStartsWith("D")
.doOnNext(person -> System.out.println(person.toString())).blockLast();
}
public static void main(String[] args) {
ApplicationContext ctx =
new AnnotationConfigApplicationContext(TestRepositoryConfig.class);
ReactiveRepositoryExample repositoryExample = new ReactiveRepositoryExample(ctx);
repositoryExample.setUp();
repositoryExample.executeRepositoryCall();
repositoryExample.cleanUp();
}
}
Restrictions
ReactiveAerospikeRepository
currently does not support the next operations:
-
all operations with indexes (create, delete, exists)
-
count()
-
deleteAll()
This limitation is due to the lack of corresponding asynchronous methods in the Aerospike client.
Projections with Aerospike
Spring Data Aerospike supports Projections, a mechanism that allows you to fetch only relevant fields from Aerospike for a particular use case. This results in better performance, less network traffic, and a better understanding of what is required for the rest of the flow.
For more details, refer to Spring Data documentation: Projections.
For example, consider a Person class:
@AllArgsConstructor
@NoArgsConstructor
@Data
@Document
public class Person {
public enum Gender {
MALE, FEMALE;
}
@Id
private long id;
private String firstName;
@Indexed(name = "lastName_idx", type = IndexType.STRING)
private String lastName;
@Field("dob")
private Date dateOfBirth;
private long heightInCm;
private boolean enabled;
private Gender gender;
private String hairColor;
private String eyeColor;
private String passportNo;
private String passptCnty;
}
This is a moderately complex object, and a production object is likely to be more complex. The use case might call for a search box that shows the firstName
, lastName
and dateOfBirth
fields, allowing the user to select a Person
based on the criteria upon which the full object will be shown.
A simple projection of this object might be:
@Data
@Builder
public class SearchPerson {
private String firstName;
private String lastName;
@Field("dob")
private Date dateOfBirth;
}
To tell Spring Data how to create a SearchPerson
it is necessary to create a method on the Person
class:
public SearchPerson toSearchPerson() {
return SearchPerson.builder()
.firstName(this.getFirstName())
.lastName(this.getLastName())
.dateOfBirth(this.getDateOfBirth())
.build();
}
Now the repository interface can be extended to return this projection:
public interface PersonRepository extends AerospikeRepository<Person, Long> {
public List<Person> findByLastName(String lastName);
public List<SearchPerson> findSearchPersonByLastName(String lastName);
}
Notice that the method name now dictates the return type of SearchPerson
as well as changing the return value. When this method is executed, Aerospike loads the full Person
objects out of storage, invokes the toSearchPerson
on each person and returns the resulting SearchPerson
instances. This reduces the required network bandwidth to present these objects to the front end and simplifies logic.
A blog post with more details on projections can be found here.
Query Methods
Spring Data Aerospike supports defining queries by method name in the Repository interface so that the implementation is generated. The format of method names is fairly flexible, comprising a verb and criteria.
Some of the verbs include find
, query
, read
, get
, count
and delete
.
For example, findByFirstName
, countByLastName
etc.
For more details, refer to basic Spring Data documentation: Defining Query Methods.
Repository Query Keywords
Here are the references to the examples of repository queries:
Note
|
Id repository read queries (like findById() , findByIds() , findByFirstNameAndId() , findAllById() , countById() , existsById() etc.) utilize get() operation of the underlying Java client. Repository read queries without id (like findByFirstName() , findByFirstNameAndLastName() , findAll() etc.) utilize query() operation of the underlying Java client.
|
Repository Interface Example
Below is an example of an interface with several query methods:
public interface PersonRepository extends AerospikeRepository<Person, Long> {
List<Person> findByLastName(String lastName);
List<Person> findByLastNameContaining(String lastName);
List<Person> findByLastNameStartingWith(String lastName);
List<Person> findByLastNameAndFirstNameContaining(String lastName, String firstName);
List<Person> findByAgeBetween(long startAge, long endAge);
Optional<Person> findById(Long id);
}
Simple Property Repository Queries
Note
|
Repository read queries without id utilize query() operation of the underlying Java client.
|
Keyword | Repository query sample | Snippet | Notes |
---|---|---|---|
Is, Equals or no keyword |
|
…where x.lastName = ? |
|
Not, IsNot |
|
…where x.lastName <> ? |
|
True, isTrue |
|
…where x.enabled = true |
|
False, isFalse |
|
…where x.enabled = false |
|
In, IsIn |
|
…where x.lastName in ? |
|
NotIn, IsNotIn |
|
…where x.lastName not in ? |
|
Null, IsNull |
|
…where x.emailAddress = null or x.emailAddress does not exist |
The same as "does not exist", objects and fields exist in AerospikeDB when their value is not equal to null. |
Exists NotNull, IsNotNull |
|
…where x.emailAddress != null |
"Exists" and "IsNotNull" represent the same functionality and can be used interchangeably, objects and fields exist in AerospikeDB when their value is not equal to null. |
LessThan, IsLessThan |
|
…where x.age < ? …where x.firstName < ? |
Strings are compared by order of each byte, assuming they have UTF-8 encoding. See information about ordering. |
LessThanEqual, IsLessThanEqual |
|
…where x.age < = ? …where x.firstName < = ? |
Strings are compared by order of each byte, assuming they have UTF-8 encoding. See information about ordering. |
GreaterThan, IsGreaterThan |
|
…where x.age > ? …where x.firstName > ? |
Strings are compared by order of each byte, assuming they have UTF-8 encoding. See information about ordering. |
GreaterThanEqual, IsGreaterThanEqual |
|
…where x.age >= ? …where x.firstName >= ? |
Strings are compared by order of each byte, assuming they have UTF-8 encoding. See information about ordering. |
Between, IsBetween |
|
…where x.age between ? and ? …where x.firstName between ? and ? |
Strings are compared by order of each byte, assuming they have UTF-8 encoding. See information about ordering. |
Before, IsBefore |
|
…where x.dateOfBirth < ? |
|
After, IsAfter |
|
…where x.dateOfBirth > ? |
|
StartingWith, IsStartingWith, StartsWith |
|
…where x.lastName like 'abc%' |
|
EndingWith, IsEndingWith, EndsWith |
|
…where x.lastName like '%abc' |
|
Like, IsLike, MatchesRegex |
|
…where x.lastName like ? |
|
Containing, IsContaining, Contains |
|
…where x.lastName like '%abc%' |
|
NotContaining, IsNotContaining, NotContains |
|
…where x.lastName not like '%abc%' |
|
And |
|
…where x.lastName = ? and x.firstName = ? |
|
Or |
|
…where x.lastName = ? or x.firstName = ? |
Collection Repository Queries
Note
|
Repository read queries without id utilize query() operation of the underlying Java client.
|
Keyword | Repository query sample | Snippet | Notes |
---|---|---|---|
Is, Equals or no keyword |
|
…where x.stringList = ? |
|
Not, IsNot |
|
…where x.stringList <> ? |
|
In |
|
…where x.stringList in ? |
Find records where |
Not In |
|
…where x.stringList not in ? |
Find records where |
Null, IsNull |
|
…where x.stringList = null or x.stringList does not exist |
The same as "does not exist", objects and fields exist in AerospikeDB when their value is not equal to null. |
Exists NotNull, IsNotNull |
|
…where x.stringList != null |
("Exists" and "IsNotNull" represent the same functionality and can be used interchangeably, objects and fields exist in AerospikeDB when their value is not equal to null). |
LessThan, IsLessThan |
|
…where x.stringList < ? |
Find records where |
LessThanEqual, IsLessThanEqual |
|
…where x.stringList < = ? |
Find records where |
GreaterThan, IsGreaterThan |
|
…where x.stringList > ? |
Find records where |
GreaterThanEqual, IsGreaterThanEqual |
|
…where x.stringList >= ? |
Find records where |
Between, IsBetween |
|
…where x.stringList between ? and ? |
Find records where |
Containing, IsContaining, Contains |
|
…where x.stringList contains ? |
|
NotContaining, IsNotContaining, NotContains |
|
…where x.stringList not contains ? |
|
And |
|
…where x.stringList = ? and x.intList = ? |
|
Or |
|
…where x.stringList = ? or x.intList = ? |
Map Repository Queries
Note
|
Repository read queries without id utilize query() operation of the underlying Java client.
|
Keyword | Repository query sample | Snippet | Notes |
---|---|---|---|
Is, Equals or no keyword |
|
…where x.stringMap = ? |
|
Not, IsNot |
|
…where x.stringMap <> ? |
|
In |
|
…where x.stringMap in ? |
Find records where |
Not In |
|
…where x.stringMap not in ? |
Find records where |
Null, IsNull |
|
…where x.stringMap = null or x.stringMap does not exist |
The same as "does not exist", objects and fields exist in AerospikeDB when their value is not equal to null. |
Exists NotNull, IsNotNull |
|
…where x.stringMap != null |
"Exists" and "IsNotNull" represent the same functionality and can be used interchangeably, objects and fields exist when their value is not equal to null. |
LessThan, IsLessThan |
|
…where x.stringMap < ? |
Find records where |
LessThanEqual, IsLessThanEqual |
|
…where x.stringMap < = ? |
Find records where |
GreaterThan, IsGreaterThan |
|
…where x.stringMap > ? |
Find records where |
GreaterThanEqual, IsGreaterThanEqual |
|
…where x.stringMap >= ? |
Find records where |
Between, IsBetween |
|
…where x.stringMap between ? and ? |
Find records where |
Containing, IsContaining, Contains |
|
…where x.stringMap contains ? |
|
NotContaining, IsNotContaining, NotContains |
|
…where x.stringMap not contains ? |
|
And |
|
…where x.stringMap = ? and x.intMap = ? |
|
Or |
|
…where x.stringMap = ? or x.intMap = ? |
POJO Repository Queries
Note
|
Repository read queries without id utilize query() operation of the underlying Java client.
|
Keyword | Repository query sample | Snippet | Notes |
---|---|---|---|
Is, Equals or no keyword |
|
…where x.address = ? |
|
Not, IsNot |
|
…where x.address <> ? |
|
In |
|
…where x.address in ? |
Find records where |
Not In |
|
…where x.address not in ? |
Find records where |
Null, IsNull |
|
…where x.address = null or x.address does not exist |
The same as "does not exist", objects and fields exist in AerospikeDB when their value is not equal to null. |
Exists NotNull, IsNotNull |
|
…where x.address != null |
"Exists" and "IsNotNull" represent the same functionality and can be used interchangeably, objects and fields exist when their value is not equal to null. |
LessThan, IsLessThan |
|
…where x.address < ? |
Find records where |
LessThanEqual, IsLessThanEqual |
|
…where x.address < = ? |
Find records where |
GreaterThan, IsGreaterThan |
|
…where x.address > ? |
Find records where |
GreaterThanEqual, IsGreaterThanEqual |
|
…where x.address >= ? |
Find records where |
Between, IsBetween |
|
…where x.address between ? and ? |
Find records where |
And |
|
…where x.address = ? and x.friend = ? |
|
Or |
|
…where x.address = ? or x.friend = ? |
Id Repository Queries
Id repository reading queries (like findById()
, findByIds()
, findByFirstNameAndId()
, findAllById()
, countById()
, existsById()
etc.) utilize get
operation of the underlying Java client (client.get()
).
Keyword | Repository query sample | Snippet | Notes |
---|---|---|---|
no keyword |
|
…where x.PK = ? |
|
And |
|
…where x.PK = ? and x.firstName = ? |
Combined Query Methods
In Spring Data, complex query methods using And
or Or
conjunction allow developers to define custom database queries based on method names that combine multiple conditions. These methods leverage query derivation, enabling developers to create expressive and type-safe queries by simply defining method signatures.
For more details, see Defining Query Methods.
For instance, a method like findByFirstNameAndLastName
will fetch records matching both conditions, while findByFirstNameOrLastName
will return records that match either condition. These query methods simplify database interaction by reducing boilerplate code and relying on convention over configuration for readability and maintainability.
In Spring Data Aerospike you define such queries by adding query methods signatures to a Repository
, as you would typically, wrapping each query parameter with QueryParam.of()
method. This method is required to pass arguments to each part of a combined query, it can receive one or more objects of the same type.
This way QueryParam
stores arguments passed to each part of a combined repository query, e.g., repository.findByNameAndEmail(QueryParam.of("John"), QueryParam.of("email"))
.
Here are some examples:
public interface CustomerRepository extends AerospikeRepository<Customer, String> {
// simple query
List<Customer> findByLastName(String lastName);
// simple query
List<Customer> findByFirstName(String firstName);
// combined query with AND conjunction
List<Customer> findByEmailAndFirstName(QueryParam email, QueryParam firstName);
// combined query with AND conjunctions
List<Customer> findByIdAndFirstNameAndAge(QueryParam id, QueryParam firstName, QueryParam age);
// combined query with OR conjunction
List<Consumer> findByFirstNameOrAge(QueryParam firstName, QueryParam age);
// combined query with AND and OR conjunctions
List<Consumer> findByEmailAndFirstNameOrAge(QueryParam email, QueryParam firstName, QueryParam age);
}
@Test
void findByCombinedQuery() {
QueryParam email = QueryParam.of(dave.getEmail());
QueryParam name = QueryParam.of(carter.getFirstName());
List<Customer> customers = repository.findByEmailAndFirstName(email, name);
assertThat(customers).isEmpty();
QueryParam ids = QueryParam.of(List.of(leroi.getId(), dave.getId(), carter.getId()));
QueryParam firstName = QueryParam.of(leroi.getFirstName());
QueryParam age = QueryParam.of(leroi.getAge());
List<Customer> customers2 = repository.findByIdAndFirstNameAndAge(ids, firstName, age);
assertThat(customers).containsOnly(leroi);
}
Query Modification
Query Modifiers
Keyword | Sample | Snippet |
---|---|---|
IgnoreCase |
findByLastNameIgnoreCase |
…where UPPER(x.lastName) = UPPER(?) |
OrderBy |
findByLastNameOrderByFirstNameDesc |
…where x.lastName = ? order by x.firstName desc |
Limiting Query Results
Keyword | Sample | Snippet |
---|---|---|
First |
findFirstByAge |
select top 1 where x.age = ? |
First N |
findFirst3ByAge |
select top 3 where x.age = ? |
Top |
findTopByLastNameStartingWith |
select top 1 where x.lastName like 'abc%' = ? |
Top N |
findTop4ByLastNameStartingWith |
select top 4 where x.lastName like 'abc%' |
Distinct |
findDistinctByFirstNameContaining |
select distinct … where x.firstName like 'abc%' |
Find Using Query
User can perform a custom Query
for finding matching entities in the Aerospike database.
A Query
can be created using a Qualifier
which represents an expression.
It may contain other qualifiers and combine them using either AND
or OR
.
Qualifier
can be created for regular bins, metadata and ids (primary keys).
Below is an example of different variations:
// creating an expression "firsName is equal to John"
Qualifier firstNameEqJohn = Qualifier.builder()
.setField("firstName")
.setFilterOperation(FilterOperation.EQ)
.setValue("John")
.build();
result = repository.findUsingQuery(new Query(firstNameEqJohn));
assertThat(result).containsOnly(john);
// creating an expression "primary key is equal to person's id"
Qualifier keyEqJohnsId = Qualifier.idEquals(john.getId());
result = repository.findUsingQuery(new Query(keyEqJohnsId));
assertThat(result).containsOnly(john);
// creating an expression "since_update_time metadata value is less than 50 seconds"
Qualifier sinceUpdateTimeLt50Seconds = Qualifier.metadataBuilder()
.setMetadataField(SINCE_UPDATE_TIME)
.setFilterOperation(FilterOperation.LT)
.setValue(50000L)
.build();
result = repository.findUsingQuery(new Query(sinceUpdateTimeLt50Seconds));
assertThat(result).contains(john);
// expressions are combined using AND
result = repository.findUsingQuery(new Query(Qualifier.and(firstNameEqJohn, keyEqJohnsId, sinceUpdateTimeLt50Seconds)));
assertThat(result).containsOnly(john);
Aerospike Object Mapping
Rich mapping support is provided by the AerospikeMappingConverter
which has a rich metadata model that provides a full feature set of functionality to map domain objects to Aerospike objects. The mapping metadata model is populated using annotations on your domain objects.
However, the infrastructure is not limited to using annotations as the only source of metadata information.
The AerospikeMappingConverter
also allows you to map objects without providing any additional metadata, by following a set of conventions.
In this section, we will describe the features of the AerospikeMappingConverter
, how to use conventions for mapping objects to documents and how to override those conventions with annotation-based mapping metadata.
For more details, refer to Spring Data documentation: Object Mapping.
Convention Based Mapping
AerospikeMappingConverter
has a few conventions for mapping objects to documents when no additional mapping metadata is provided.
The conventions are:
How the 'id' Field Is Handled in the Mapping Layer
Aerospike DB requires that you have an id
field for all objects.
The id
field can be of any primitive type as well as String
or byte[]
.
The following table outlines the requirements for the id
field:
Field definition | Description |
---|---|
|
A field named 'id' without an annotation |
|
A field annotated with |
The following description outlines what type of conversion, if any, will be done on the property mapped to the id
document field:
-
By default, the type of the field annotated with
@id
is turned into aString
to be stored in Aerospike database. If the original type cannot be persisted (see keepOriginalKeyTypes for details), it must be convertible toString
and will be stored in the database as such, then converted back to the original type when the object is read. This is transparent to the application but needs to be considered if using external tools likeAQL
to view the data. -
If no field named "id" is present in the Java class then an implicit '_id' file will be generated by the driver but not mapped to a property or field of the Java class.
When querying and updating AerospikeTemplate
will use the converter to handle conversions of the Query
and Update
objects that correspond to the above rules for saving documents so field names and types used in your queries will be able to match what is in your domain classes.
Mapping Configuration
Unless explicitly configured, an instance of AerospikeMappingConverter
is created by default when creating a AerospikeTemplate
.
You can create your own instance of the MappingAerospikeConverter
so as to tell it where to scan the classpath at the startup of your domain classes in order to extract metadata and construct indexes.
Also, to have more control over the conversion process (if needed), you can register converters to use for mapping specific classes to and from the database.
Note
|
AbstractAerospikeConfiguration will create an AerospikeTemplate instance and register with the container under the name 'AerospikeTemplate'. |
Mapping Annotation Overview
The MappingAerospikeConverter can use metadata to drive the mapping of objects to documents using annotations. An overview of the annotations is provided below
-
@Id
- applied at the field level to mark the field used for identity purposes. -
@Field
- applied at the field level, describes the name of the field as it will be represented in the AerospikeDB BSON document thus allowing the name to be different from the field name of the class. -
@Version
- applied at the field level to mark record modification count. The value must be effectively integer. In Spring Data Aerospike, documents come in two forms – non-versioned and versioned. Documents with an@Version
annotation have a version field populated by the corresponding record’s generation count. Version can be passed to a constructor or not (in that case it stays equal to zero). -
@Expiration
- applied at the field level to mark a property to be used as expiration field. Expiration can be specified in two flavors: as an offset in seconds from the current time (then field value must be effectively integer) or as an absolute Unix timestamp. Client system time must be synchronized with Aerospike server system time, otherwise expiration behaviour will be unpredictable.
The mapping metadata infrastructure is defined in a separate spring-data-commons project that is technology-agnostic. Specific subclasses are used in the AerospikeDB support to support annotation-based metadata. Other strategies are also possible to put in place if there is demand.
Here is an example of a more complex mapping.
public class Person<T extends Address> {
@Id
private String id;
private Integer ssn;
@Field("fName")
private String firstName;
private String lastName;
private Integer age;
private Integer accountTotal;
private List<Account> accounts;
private T address;
@Version
private int id; // must be integer
public Person(Integer ssn) {
this.ssn = ssn;
}
public Person(Integer ssn, String firstName, String lastName, Integer age, T address, int version) {
this.ssn = ssn;
this.firstName = firstName;
this.lastName = lastName;
this.age = age;
this.address = address;
this.version = version;
}
public String getId() {
return id;
}
// no setter for Id. (getter is only exposed for some unit testing)
public Integer getSsn() {
return ssn;
}
// other getters/setters omitted
}
Aerospike Custom Converters
Spring type converters are components used to convert data between different types, particularly when interacting with databases or binding data from external sources. They facilitate seamless transformation of data, such as converting between String and database-specific types (e.g., LocalDate to DATE or String to enumerations).
For more details, see Spring Type Conversion.
Spring provides a set of default type converters for common conversions. Spring Data Aerospike has its own built-in converters in DateConverters
and AerospikeConverters
classes.
However, in certain cases, custom converters are necessary to handle specific logic or custom serialization requirements. Custom converters allow developers to define precise conversion rules, ensuring data integrity and compatibility between application types and database representations.
In order to add a custom converter you can leverage Spring’s Converter
SPI to implement type conversion logic and override customConverters()
method available in AerospikeDataConfigurationSupport
. Here is an example:
public class BlockingTestConfig extends AbstractAerospikeDataConfiguration {
@Override
protected List<Object> customConverters() {
return List.of(
CompositeKey.CompositeKeyToStringConverter.INSTANCE,
CompositeKey.StringToCompositeKeyConverter.INSTANCE
);
}
@Value
public static class CompositeKey {
String firstPart;
long secondPart;
@WritingConverter
public enum CompositeKeyToStringConverter implements Converter<CompositeKey, String> {
INSTANCE;
@Override
public String convert(CompositeKey source) {
return source.firstPart + "::" + source.secondPart;
}
}
@ReadingConverter
public enum StringToCompositeKeyConverter implements Converter<String, CompositeKey> {
INSTANCE;
@Override
public CompositeKey convert(String source) {
String[] split = source.split("::");
return new CompositeKey(split[0], Long.parseLong(split[1]));
}
}
}
}
Aerospike Template
Aerospike Template provides a set of features for interacting with the database. It allows lower-level access than a Repository and also serves as the foundation for repositories.
Template is the central support class for Aerospike database operations. It provides the following functionality:
-
Methods to interact with the database
-
Mapping between Java objects and Aerospike Bins (see Object Mapping)
-
Providing connection callback
-
Translating exceptions into Spring’s technology-agnostic DAO exceptions hierarchy
Instantiating AerospikeTemplate
If you are subclassing AbstractAerospikeDataConfiguration
then the aerospikeTemplate
bean is already present in your context, and you can use it.
@Autowired
protected AerospikeTemplate template;
An alternative is to instantiate it yourself, you can see the bean in AbstractAerospikeDataConfiguration
.
In case if you need to use custom WritePolicy
, the persist
operation can be used.
For CAS updates save
operation must be used.
Methods for interacting with database
AerospikeOperations
interface provides operations for interacting with the database (exists
, find
, insert
, update
etc.) as well as basic operations with indexes: createIndex
, deleteIndex
, indexExists
.
The names of operations are typically self-descriptive. To read from Aerospike you can use findById
, findByIds
and find
methods, to delete - delete
methods, and so on.
template.findById(id, Person.class)
For indexed documents use find
with provided Query
object.
Stream<Person> result = template.find(query, Person.class);
assertThat(result).hasSize(6);
Example
The simple case of using the save operation is to save a POJO.
Note
|
For more information about Id property when inserting or saving see Mapping Conventions: Id Field for more information. |
public class Person {
@Id
private String id;
private String firstName;
private String lastName;
private int age;
}
template.insert(new Person(id, "John", 50));
long count = template.count
(new Query
(new QualifierBuilder()
.setFilterOperation(FilterOperation.EQ)
.setField("firstName")
.setValue("John")
.build()
),
Person.class
);
assertThat(count).isEqualTo(3);
Secondary indexes
A secondary index (SI) is a data structure that locates all the records in a namespace, or a set within it, based on a bin value in the record. When a value is updated in the indexed record, the secondary index automatically updates.
You can read more about secondary index implementation and usage in Aerospike on the official documentation page.
Why Secondary Index
Let’s consider a simple query for finding by equality:
public List<Person> personRepository.findByLastName(lastName);
Notice that findByLastName is not a simple lookup by key, but rather finding all records in a set. Aerospike has 2 ways of achieving this:
-
Scanning all the records in the set and extracting the appropriate records.
-
Defining a secondary index on the field
lastName
and using this secondary index to satisfy the query.
The second approach is far more efficient. Aerospike stores the secondary indexes in a memory structure, allowing exceptionally fast identification of the records that match.
It relies on a secondary index having been created.
Ways to Create Secondary Indexes
In SpringData Aerospike secondary indexes can either be created by systems administrators using the asadm
tool, or by developers telling SpringData that such an index is necessary.
There are two ways to accomplish this task with the help of SpringData Aerospike:
-
Using AerospikeTemplate
createIndex
method. -
Using
@Indexed
annotation on the necessary field of an entity.
Creating Secondary Index via AerospikeTemplate
For more information about AerospikeTemplate see the documentation page.
Setting a secondary index via AerospikeTemplate can be helpful, for example, in cases when an index creation does not change a lot.
Here is an example of a numeric secondary index for the rating
field in the MovieDocument
entity:
@Slf4j
@Configuration
public class AerospikeIndexConfiguration {
private static final String INDEX_NAME = "movie-rating-index";
@Bean
@ConditionalOnProperty(
value = "aerospike." + INDEX_NAME + ".create-on-startup",
havingValue = "true",
matchIfMissing = true)
public boolean createAerospikeIndex(AerospikeTemplate aerospikeTemplate) {
try {
aerospikeTemplate.createIndex(MovieDocument.class, INDEX_NAME, "rating", IndexType.NUMERIC);
log.info("Index {} was successfully created", INDEX_NAME);
} catch (Exception e) {
log.info("Index {} creation failed: {}", INDEX_NAME, e.getMessage());
}
return true;
}
}
Creating Secondary Index using @Indexed annotation
You can use @Indexed
annotation on the field where the index is required.
Here is an example of the Person
object getting indexed by lastName
:
@AllArgsConstructor
@NoArgsConstructor
@Data
@Document
public class Person {
@Id
private long id;
private String firstName;
@Indexed(name = "lastName_idx", type = IndexType.STRING)
private String lastName;
private Date dateOfBirth;
}
The annotation allows to specify also bin name, collectionType and ctx (context) if needed.
For the details on using @Indexed
annotation see Indexed Annotation.
Matching the Secondary Index
Note
|
In Aerospike, secondary indexes are case-sensitive, they match the exact queries. |
Following the query from the example above, assume there was a new requirement to be able to find by lastName
containing a String (rather than having an equality match):
public List<Person> findByLastNameContaining(String lastName);
In this case findByLastNameContaining
query is not satisfied by the created secondary index.
Aerospike would need to scan the data which can be an expensive operation as all records in the set must be read
by the Aerospike server, and then the condition is applied to see if they match.
Due to the cost of performing this operation, scans from Spring Data Aerospike are disabled by default.
For the details on how to enable scans see Scan Operation.
Following the query from the example above, assume there was a new requirement to be able to find by firstName
with an exact match:
public List<Person> findByLastName(String lastName);
public List<Person> findByFirstName(String firstName);
In this case firstName
is not marked as @Indexed
, so SpringData Aerospike is not instructed to create an index on it.
Hence, it will scan the repository (a costly operation that could be avoided by using an index).
Note
|
There are relevant configuration parameters: create indexes on startup and indexes cache refresh frequency. |
Indexed Annotation
The @Indexed
annotation allows to create secondary index based on a specific field of a Java object.
For the details on secondary indexes in Aerospike see Secondary Indexes.
The annotation allows to specify the following parameters:
parameter | index type | mandatory | example |
---|---|---|---|
name |
index name |
yes |
"friend_address_keys_idx" |
type |
index type |
yes |
IndexType.STRING |
bin |
indexed bin type |
no |
"friend" |
collectionType |
index type |
no |
IndexCollectionType.MAPKEYS |
ctx |
context (path to the indexed elements) |
no |
"address" |
Here is an example of creating a complex secondary index for fields of a person’s friend address.
@Data
@AllArgsConstructor
public class Address {
private String street;
private Integer apartment;
private String zipCode;
private String city;
}
@Data
@NoArgsConstructor
@Setter
public class Friend {
String name;
Address address;
}
@Data
@Document
@AllArgsConstructor
@NoArgsConstructor
@Setter
public class Person {
@Id
String id;
@Indexed(type = IndexType.STRING, name = "friend_address_keys_idx",
collectionType = IndexCollectionType.MAPKEYS, ctx = "address")
Friend friend;
}
@Test
void test() {
Friend carter = new Friend();
carter.setAddress(new Address("Street", 14, "1234567890", "City"));
Person dave = new Person();
dave.setFriend(carter);
repository.save(dave);
}
A Person
object in this example has a field called "friend" (Friend
object).
A Friend
object has a field called "address" (Address
object).
So when "friend" field is set to a Friend
with existing Address
, we have a person (dave
in the example above) with a friend (carter
) who has
a particular address.
Address
object on its own has certain fields: street
, apartment
, zipCode
, city
.
Note
|
In Aerospike DB a POJO (such as Address ) is represented by a Map, so the fields of POJO become map keys.
|
Thus, if we want to index by Address
object fields, we set collectionType
to IndexCollectionType.MAPKEYS
.
Ctx
parameter represents context, or path to the necessary element in the specified bin ("friend") - which is "address", because we want to index by fields of friend’s address.
Secondary Index Context DSL
Secondary index context (ctx
parameter in @Indexed
annotation) represents path to a necessary element in hierarchy. It uses infix notation.
The document path is described as dot-separated context elements (e.g., "a.b.[2].c") written as a string. A path is made of singular path elements and ends with one (a leaf element) or more elements (leaves) - for example, "a.b.[2].c.[0:3]".
Path Element | Matches | Notes |
---|---|---|
|
Map key “a” |
Single element by key |
|
Map key (numeric string) “1” |
|
|
Map key (integer) 1 |
|
|
Map index 1 |
|
|
Map value (integer) 1 |
|
|
Map value “bb” |
Also {="bb"} |
|
Map value (string) “1” |
|
|
Map rank 1 |
|
|
List index 1 |
|
|
List value 1 |
|
|
List rank 1 |
Example
Let’s consider a Map bin example:
{
1: a,
2: b,
4: d,
"5": e,
a: {
55: ee,
"66": ff,
aa: {
aaa: 111,
bbb: 222,
ccc: 333,
},
bb: {
bba: 221,
bbc: 223
},
cc: [ 22, 33, 44, 55, 43, 32, 44 ],
dd: [ {e: 5, f:6}, {z:26, y:25}, {8: h, "9": j} ]
}
}
So the following will be true:
Path | CTX | Matched Value |
---|---|---|
a.aa.aaa |
[mapKey("a"), mapKey("aa"), mapKey("aaa")] |
111 |
a.55 |
[mapKey("a"), mapKey(55)] |
ee |
a."66" |
[mapKey("a"), mapKey("66")] |
ff |
a.aa.{2} |
[mapKey("a"), mapKey("aa"),mapIndex(2)] |
333 |
a.aa.{=222} |
[mapKey("a"), mapKey("aa"),mapValue(222)] |
222 |
a.bb.{#-1} |
[mapKey("a"), mapKey("bb"),mapRank(-1)] |
223 |
a.cc.[0] |
[mapKey("a"), mapKey("cc"),listIndex(0)] |
22 |
a.cc.[#1] |
[mapKey("a"), mapKey("cc"),listRank(1)] |
32 |
a.cc.[=44] |
[mapKey("a"), mapKey("cc"),listValue(44)] |
[44, 44] |
a.dd.[0].e |
[mapKey("a"), mapKey("dd"),listIndex(0), mapKey("e")] |
5 |
a.dd.[2].8 |
[mapKey("a"), mapKey("dd"),listIndex(2), mapKey(8)] |
h |
a.dd.[-1]."9" |
[mapKey("a"), mapKey("dd"),listIndex(-1), mapKey("9")] |
j |
a.dd.[1].{#0} |
[mapKey("a"), mapKey("dd"),listIndex(1), mapRank(0)] |
y |
Note
|
There are relevant configuration parameters: create indexes on startup and indexes cache refresh frequency. |
Caching
Caching is the process of storing data in a cache or temporary storage location, usually to improve application performance and make data access faster.
The caching process also provides an efficient way to reuse previously retrieved or computed data. The cache is used to reduce the need for accessing the underlying storage layer which is slower.
Spring Cache with Aerospike database allows you to use annotations such as @Cacheable
, @CachePut
and @CacheEvict
that provide a fully managed cache store using Aerospike database.
Introduction
In this example, we are going to use the annotations on UserRepository
class methods to create/read/update and delete user’s data from the cache.
If a User
is stored in the cache, calling a method with @Cacheable
annotation will fetch the user from the cache instead of executing the method’s body responsible for the actual user fetch from the database.
If the User
does not exist in the cache, the user’s data will be fetched from the database and put in the cache for later usage (a “cache miss”).
With Spring Cache and Aerospike database, we can achieve that with only a few lines of code.
Motivation
Let’s say that we are using another database as our main data store. We don’t want to fetch the results from it every time we request the data, instead, we want to get the data from a cache layer.
There is a number of benefits of using a cache layer, here are some of them:
-
Performance: Aerospike can work purely in RAM but reading a record from Aerospike in Hybrid Memory (primary index in memory, data stored on Flash drives) is extremely fast as well (~1ms).
-
Reduce database load: Moving a significant part of the read load from the main database to Aerospike can help balance the resources on heavy loads.
-
Scalability: Aerospike scales horizontally by adding more nodes to the cluster, scaling a relational database might be tricky and expensive, so if you are facing a read heavy load you can easily scale up the cache layer.
Example
We will not use an actual database as our main data store for this example, instead, we will simulate database access by printing a simulation message and replace a database read by just returning a specific User
.
Configuration
AerospikeConfigurationProperties
@Data
@Component
@ConfigurationProperties(prefix = "aerospike")
public class AerospikeConfigurationProperties {
private String host;
private int port;
}
AerospikeConfiguration
@Configuration
@EnableConfigurationProperties(AerospikeConfigurationProperties.class)
@Import(value = {MappingAerospikeConverter.class, AerospikeMappingContext.class,
AerospikeTypeAliasAccessor.class,
AerospikeCustomConversions.class, SimpleTypeHolder.class})
public class AerospikeConfiguration {
@Autowired
private MappingAerospikeConverter mappingAerospikeConverter;
@Autowired
private AerospikeConfigurationProperties aerospikeConfigurationProperties;
@Bean(destroyMethod = "close")
public AerospikeClient aerospikeClient() {
ClientPolicy clientPolicy = new ClientPolicy();
clientPolicy.failIfNotConnected = true;
return new AerospikeClient(clientPolicy, aerospikeConfigurationProperties.getHost(),
aerospikeConfigurationProperties.getPort());
}
@Bean
public CacheManager cacheManager(IAerospikeClient aerospikeClient,
MappingAerospikeConverter aerospikeConverter,
AerospikeCacheKeyProcessor cacheKeyProcessor) {
AerospikeCacheConfiguration defaultConfiguration = new AerospikeCacheConfiguration("test");
return new AerospikeCacheManager(aerospikeClient, mappingAerospikeConverter, defaultConfiguration,
cacheKeyProcessor);
}
}
In the AerospikeConfiguration we will create two types of Beans:
AerospikeClient
Responsible for accessing an Aerospike database and performing database operations.
AerospikeCacheManager
The heart of the cache layer, to define an AerospikeCacheManager you need:
-
aerospikeClient (AerospikeClient).
-
aerospikeConverter (MappingAerospikeConverter).
-
defaultCacheConfiguration (AerospikeCacheConfiguration), a default cache configuration that applies when creating new caches. Cache configuration contains a namespace, a set (null by default meaning write directly to the namespace w/o specifying a set) and an expirationInSeconds (AKA TTL, default is 0 meaning use Aerospike server’s default).
-
Optional: initialPerCacheConfiguration (Map<String, AerospikeCacheConfiguration>), You can also specify a map of cache names and matching configuration, it will create the caches with the given matching configuration at the application startup.
Note
|
A cache name is only a link to the cache configuration. |
Objects
User
@Data
@Document
@AllArgsConstructor
public class User {
@Id
private int id;
private String name;
private String email;
private int age;
}
Repositories
UserRepository
@Repository
public class UserRepository {
@Cacheable(value = "test", key = "#id")
public Optional<User> getUserById(int id) {
System.out.println("Simulating a read from the main data store.");
// In case the id doesn't exist in the cache it will "fetch" jimmy page with the requested id and add it to the cache (cache miss).
return Optional.of(new User(id, "jimmy page", "jimmy@gmail.com", 77));
}
@CachePut(value = "test", key = "#user.id")
public User addUser(User user) {
System.out.println("Simulating addition of " + user + " to the main data store.");
return user;
}
@CacheEvict(value = "test", key = "#id")
public void removeUserById(int id) {
System.out.println("Simulating removal of " + id + " from the main data store.");
}
}
The cache annotations require a “value” field, which is the cache name, if the cache name doesn’t exist — by passing initialPerCacheConfiguration param when creating a Bean of AerospikeCacheManager in a configuration class, it will configure the cache with the properties of the given defaultCacheConfiguration (Configuration > AerospikeCacheManager).
Services
UserService
@Service
@AllArgsConstructor
public class UserService {
UserRepository userRepository;
public Optional<User> readUserById(int id) {
return userRepository.getUserById(id);
}
public User addUser(User user) {
return userRepository.addUser(user);
}
public void removeUserById(int id) {
userRepository.removeUserById(id);
}
}
Controllers
UserController
@RestController
@AllArgsConstructor
public class UserController {
UserService userService;
@GetMapping("/users/{id}")
public Optional<User> readUserById(@PathVariable("id") Integer id) {
return userService.readUserById(id);
}
@PostMapping("/users")
public User addUser(@RequestBody User user) {
return userService.addUser(user);
}
@DeleteMapping("/users/{id}")
public void deleteUserById(@PathVariable("id") Integer id) {
userService.removeUserById(id);
}
}
Add @EnableCaching
SimpleSpringBootAerospikeCacheApplication
Add @EnableCaching
to the class that contains the main method.
@EnableCaching
@SpringBootApplication
public class SimpleSpringBootAerospikeCacheApplication {
public static void main(String[] args) {
SpringApplication.run(SimpleSpringBootAerospikeCacheApplication.class, args);
}
}
Test
We will use Postman to simulate client requests.
Add User (@CachePut)
-
Create a new POST request with the following url: http://localhost:8080/users
-
Add a new key-value header in the Headers section:
Key: Content-Type
Value: application/json
-
Add a Body in a valid JSON format:
{ "id":1, "name":"guthrie", "email":"guthriegovan@gmail.com", "age":35 }
-
Press Send.
aql> select * from test
+-----+-----------+----------+-------------+-------------------------------------+
| @user_key | name | @_class | email | age |
+-----+-----------+----------+-------------+-------------------------------------+
| "1" | "guthrie" | "com.aerospike.cache.simpleSpringBootAerospikeCache.objects.User" | "guthriegovan@gmail.com" | 35 |
+-----+-----------+----------+-------------+-------------------------------------+
We can now see that this user was added to the cache.
Read User (@Cacheable)
-
Create a new GET request with the following url: http://localhost:8080/users/1
-
Add a new key-value header in the Headers section:
Key: Content-Type
Value: application/json
-
Press Send.
Remove User (@CacheEvict)
-
Create a new DELETE request with the following url: http://localhost:8080/users/1
-
Add a new key-value header in the Headers section:
Key: Content-Type
Value: application/json
-
Press Send.
We can now see that this user was deleted from the cache (thanks to the @CacheEvict annotation in the UserRepository).
aql> select * from test
+-----+-----------+----------+-------------+-------------------------------------+
0 rows in set
+-----+-----------+----------+-------------+-------------------------------------+
Cache miss (@Cacheable)
For reading User
that is not in the cache we can use the GET request configured before with an id that we know for sure is not there.
If we try calling the GET request with the id 5, we get the following user data:
{ "id": 5, "name": "jimmy page", "email": "jimmy@gmail.com", "age": 77 }
We wrote it hard-coded in UserRepository
to simulate an actual database fetch of a user id that doesn’t exist in the cache.
We can now also see that the user was added to the cache.
aql> select * from test
+-----+-----------+----------+-------------+-------------------------------------+
| @user_key | name | @_class | email | age |
+-----+-----------+----------+-------------+-------------------------------------+
| "1" | "jimmy page" | "com.aerospike.cache.simpleSpringBootAerospikeCache.objects.User" | "jimmy@gmail.com" | 77 |
+-----+-----------+----------+-------------+-------------------------------------+
Configuration
Configuration parameters can be set in a standard application.properties
file using spring.data.aerospike*
prefix
or by overriding configuration from AbstractAerospikeDataConfiguration
class.
Application.properties
Here is an example:
# application.properties
spring.aerospike.hosts=localhost:3000
spring.data.aerospike.namespace=test
spring.data.aerospike.scans-enabled=false
spring.data.aerospike.send-key=true
spring-data-aerospike.create-indexes-on-startup=true
spring.data.aerospike.index-cache-refresh-seconds=3600
spring.data.aerospike.server-version-refresh-seconds=3600
spring.data.aerospike.query-max-records=10000
spring.data.aerospike.batch-write-size=100
spring.data.aerospike.keep-original-key-types=false
Configuration class:
@Configuration
@EnableAerospikeRepositories(basePackageClasses = {TestRepository.class})
public class AerospikeConfiguration extends AbstractAerospikeDataConfiguration {
}
In this case extending AbstractAerospikeDataConfiguration
class is required to enable repositories.
Overriding configuration
Configuration can also be set by overriding getHosts()
, nameSpace()
and configureDataSettings()
methods of the AbstractAerospikeDataConfiguration
class.
Here is an example:
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected Collection<Host> getHosts() {
return Collections.singleton(new Host("localhost", 3000));
}
@Override
protected String nameSpace() {
return "test";
}
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setScansEnabled(false);
aerospikeDataSettings.setCreateIndexesOnStartup(true);
aerospikeDataSettings.setIndexCacheRefreshSeconds(3600);
aerospikeDataSettings.setServerVersionRefreshSeconds(3600);
aerospikeDataSettings.setQueryMaxRecords(10000L);
aerospikeDataSettings.setBatchWriteSize(100);
aerospikeDataSettings.setKeepOriginalKeyTypes(false);
}
}
Note
|
Return values of getHosts() , nameSpace() and configureDataSettings() methods
of the AbstractAerospikeDataConfiguration class have precedence over the parameters
set via application.properties.
|
Configuration Parameters
hosts
# application.properties
spring.aerospike.hosts=hostname1:3001, hostname2:tlsName2:3002
A String of hosts separated by ,
in form of hostname1[:tlsName1][:port1],…
IP addresses must be given in one of the following formats:
IPv4: xxx.xxx.xxx.xxx
IPv6: [xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx]
IPv6: [xxxx::xxxx]
IPv6 addresses must be enclosed by brackets. tlsName is optional.
Note
|
Another way of defining hosts is overriding the getHosts() method.
It has precedence over hosts parameter from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected Collection<Host> getHosts() {
return Collections.singleton(new Host("hostname1", 3001));
}
}
Default: null
.
namespace
# application.properties
spring.data.aerospike.namespace=test
Aerospike DB namespace.
Note
|
Another way of defining hosts is overriding the nameSpace() method.
It has precedence over namespace parameter from application.properties.
Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected String nameSpace() {
return "test";
}
}
Note
|
To use multiple namespaces it is required to override nameSpace() and AerospikeTemplate for each
configuration class per namespace.
See multiple namespaces example
for implementation details.
|
Default: null
.
scansEnabled
# application.properties
spring.data.aerospike.scans-enabled=false
A scan can be an expensive operation as all records in the set must be read by the Aerospike server, and then the condition is applied to see if they match.
Due to the cost of performing this operation, scans from Spring Data Aerospike are disabled by default.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setScansEnabled(false);
}
}
Note
|
Once this flag is enabled, scans run whenever needed with no warnings. This may or may not be optimal in a particular use case. |
Default: false
.
createIndexesOnStartup
# application.properties
spring.data.aerospike.create-indexes-on-startup=true
Create secondary indexes specified using @Indexed
annotation on startup.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setCreateIndexesOnStartup(true);
}
}
Default: true
.
indexCacheRefreshSeconds
# application.properties
spring.data.aerospike.index-cache-refresh-seconds=3600
Automatically refresh indexes cache every <N> seconds.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setIndexCacheRefreshSeconds(3600);
}
}
Default: 3600
.
serverVersionRefreshSeconds
# application.properties
spring.data.aerospike.server-version-refresh-seconds=3600
Automatically refresh cached server version every <N> seconds.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setServerVersionRefreshSeconds(3600);
}
}
Default: 3600
.
queryMaxRecords
# application.properties
spring.data.aerospike.query-max-records=10000
Limit amount of results returned by server. Non-positive value means no limit.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setQueryMaxRecords(10000L);
}
}
Default: 10 000
.
batchWriteSize
# application.properties
spring.data.aerospike.batch-write-size=100
Maximum batch size for batch write operations. Non-positive value means no limit.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setBatchWriteSize(100);
}
}
Default: 100
.
keepOriginalKeyTypes
# application.properties
spring.data.aerospike.keep-original-key-types=false
Define how @Id
fields (primary keys) and Map
keys are stored in the Aerospike database:
false
- always as String
, true
- preserve original type if supported.
@Id field type |
keepOriginalKeyTypes = false |
keepOriginalKeyTypes = true |
---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
other types |
|
|
Note
|
If @Id field’s type cannot be persisted as is, it must be convertible to String and will be stored
in the database as such, then converted back to the original type when the object is read.
This is transparent to the application but needs to be considered if using external tools like AQL to view the data.
|
Map key type |
keepOriginalKeyTypes = false |
keepOriginalKeyTypes = true |
---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
other types |
|
|
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setKeepOriginalKeyTypes(false);
}
}
Default: false
(store keys only as String
).
writeSortedMaps
# application.properties
spring.data.aerospike.writeSortedMaps=true
Define how Maps and POJOs are written: true
- as sorted maps (TreeMap
, default), false
- as unsorted (HashMap
).
Writing as unsorted maps (false
) degrades performance of Map-related operations and does not allow comparing Maps,
so it is strongly recommended to change the default value only if required during upgrade from older versions
of Spring Data Aerospike.
Note
|
Another way of defining the parameter is overriding the configureDataSettings() method.
It has precedence over reading from application.properties. Here is an example:
|
// overriding method
@EnableAerospikeRepositories(basePackageClasses = TestRepository.class)
class ApplicationConfig extends AbstractAerospikeDataConfiguration {
@Override
protected void configureDataSettings(AerospikeDataSettings aerospikeDataSettings) {
aerospikeDataSettings.setWriteSortedMaps(true);
}
}
Default: true
(write Maps and POJOs as sorted maps).