If you are implementing database sharding and using Spring JDBC then you are out of luck to using declarative transactions and find a Datasource with Spring that would handle sharding. I had to implement my own Datasource manager and own annotations to use declarative kind of transactions to hide complexities from average developers. Its very important to abstract out cross cutting concerns as sharding and transactions so that any junior developers wont be confused and start copying code left and right without understanding the impact of their changes globally.
So the idea is that
1) You would implement a ShardDataSourceManager that would be basically pool of connection pools and you would lookup a datasource by shard id.
2)You would define your own Transactional annotations and annotate methods with it
3) You need to write an interceptor at dao layer that would read annotations on method and some context info. From the context info you would lookup shard id and lookup datasource and inject into a thread local.
4)The dao layer when it looks up datasource would look into thread local to construct a jdbc template and execute queries on it.
Here is a sample ShardDataSourceManager, ShardTransactional Annotation
public @interface ShardTransactional {
public abstract boolean readOnly() default false;
}
public class ShardTransactionInterceptor implements MethodInterceptor {
private static final AppLogger logger = AppLogger.getLogger(ShardTransactionInterceptor.class);
private static ThreadLocal dataSourceThreadLocal = new ThreadLocal();
private ShardDataSourceManager shardDataSourceManager;
public ShardDataSourceManager getShardDataSourceManager() {
return shardDataSourceManager;
}
public void setShardDataSourceManager(ShardDataSourceManager shardDataSourceManager) {
this.shardDataSourceManager = shardDataSourceManager;
}
@Override
public Object invoke(final MethodInvocation method) throws Throwable {
if (method.getMethod().isAnnotationPresent(ShardTransactional.class)) {
try {
ShardTransactional annotation = method.getMethod().getAnnotation(ShardTransactional.class);
User user = getParam(method, User.class);
if (user == null) {
throw new IllegalStateException("All transactional methods must have user argument");
}
TransactionTemplate transactionTemplate = new TransactionTemplate();
boolean readOnly = annotation.readOnly();
transactionTemplate.setReadOnly(readOnly);
ShardInfo shardInfo = getShardInfo(user);
transactionTemplate.setName("ShardTransaction");
transactionTemplate.setTransactionManager(shardDataSourceManager.getTransactionManagerByHostId(shardInfo.getHostId(), readOnly));
cacheDataSourceInThreadLocal(shardInfo.getHostId(),readOnly);
return transactionTemplate.execute(new TransactionCallback
So the idea is that
1) You would implement a ShardDataSourceManager that would be basically pool of connection pools and you would lookup a datasource by shard id.
2)You would define your own Transactional annotations and annotate methods with it
3) You need to write an interceptor at dao layer that would read annotations on method and some context info. From the context info you would lookup shard id and lookup datasource and inject into a thread local.
4)The dao layer when it looks up datasource would look into thread local to construct a jdbc template and execute queries on it.
Here is a sample ShardDataSourceManager, ShardTransactional Annotation
public @interface ShardTransactional {
public abstract boolean readOnly() default false;
}
public class ShardTransactionInterceptor implements MethodInterceptor {
private static final AppLogger logger = AppLogger.getLogger(ShardTransactionInterceptor.class);
private static ThreadLocal
private ShardDataSourceManager shardDataSourceManager;
public ShardDataSourceManager getShardDataSourceManager() {
return shardDataSourceManager;
}
public void setShardDataSourceManager(ShardDataSourceManager shardDataSourceManager) {
this.shardDataSourceManager = shardDataSourceManager;
}
@Override
public Object invoke(final MethodInvocation method) throws Throwable {
if (method.getMethod().isAnnotationPresent(ShardTransactional.class)) {
try {
ShardTransactional annotation = method.getMethod().getAnnotation(ShardTransactional.class);
User user = getParam(method, User.class);
if (user == null) {
throw new IllegalStateException("All transactional methods must have user argument");
}
TransactionTemplate transactionTemplate = new TransactionTemplate();
boolean readOnly = annotation.readOnly();
transactionTemplate.setReadOnly(readOnly);
ShardInfo shardInfo = getShardInfo(user);
transactionTemplate.setName("ShardTransaction");
transactionTemplate.setTransactionManager(shardDataSourceManager.getTransactionManagerByHostId(shardInfo.getHostId(), readOnly));
cacheDataSourceInThreadLocal(shardInfo.getHostId(),readOnly);
return transactionTemplate.execute(new TransactionCallback
Seems like a neat solution. However as I observed sharding eventaully becomes much more than just inserts in a "shard-aware" connection pool. Cross-shard queries, transaction consistency and administration of the entire array - are crucial to have a a good sharding solution. You can have a look at ScaleBase (disclaimer: I work there), http://www.scalebase.com, to see how a this can be your 1-stop-shop for all of your sharding needs, totally transparent (standard conn pool... :) ).
ReplyDeleteCan I get the source code for this to play with?
ReplyDeleteexcept the imports the code pasted above is the real source code we have live in production serving 1B+ rows from 20 mysql servers. I havent got a chance to put it on github yet.
ReplyDeleteAny github project ? looks nice, i'm doing similar stuff and i'd like to fork and contribute if possible
ReplyDeleteNo github project right now :( as I got busy.
Delete