API Documentation¶
Configuration¶
sherlock
can be globally configured to set a lot of defaults using the
sherlock.configure()
function. The configuration set using
sherlock.configure()
will be used as the default for all the lock
objects.
-
sherlock.
configure
(**kwargs)[source]¶ Set basic global configuration for
sherlock
.Parameters: - backend – global choice of backend. This backend will be used
for managing locks by
sherlock.Lock
class objects. - client – global client object to use to connect with backend
store. This client object will be used to connect to the
backend store by
sherlock.Lock
class instances. The client object must be a valid object of the client library. If the backend has been configured using the backend parameter, the custom client object must belong to the same library that is supported for that backend. If the backend has not been set, then the custom client object must be an instance of a valid supported client. In that case,sherlock
will set the backend by introspecting the type of provided client object. - namespace (str) – provide global namespace
- expire (float) – provide global expiration time. If expicitly set to None, lock will not expire.
- timeout (float) – provide global timeout period
- retry_interval (float) – provide global retry interval
Basic Usage:
>>> import sherlock >>> from sherlock import Lock >>> >>> # Configure sherlock to use Redis as the backend and the timeout for >>> # acquiring locks equal to 20 seconds. >>> sherlock.configure(timeout=20, backend=sherlock.backends.REDIS) >>> >>> import redis >>> redis_client = redis.StrictRedis(host='X.X.X.X', port=6379, db=1) >>> sherlock.configure(client=redis_client)
- backend – global choice of backend. This backend will be used
for managing locks by
Understanding the configuration parameters¶
sherlock.configure()
accepts certain configuration patterns which are
individually explained in this section. Some of these configurations are global
configurations and cannot be overriden while others can be overriden at
individual lock levels.
backend
¶
The backend parameter allows you to set which backend you would like to use
with the sherlock.Lock
class. When set to a particular backend,
instances of sherlock.Lock
will use this backend for lock
synchronization.
Basic Usage:
>>> import sherlock
>>> sherlock.configure(backend=sherlock.backends.REDIS)
Note
this configuration cannot be overriden at the time of creating a class object.
Available Backends¶
To set the backend global configuration, you would have to choose one from the defined backends. The defined backends are:
- Etcd:
sherlock.backends.ETCD
- Memcache:
sherlock.backends.MEMCACHE
- Redis:
sherlock.backends.REDIS
client
¶
The client parameter allows you to set a custom clien object which
sherlock
can use for connecting to the backend store. This gives you the
flexibility to connect to the backend store from the client the way you want.
The provided custom client object must be a valid client object of the
supported client libraries. If the global backend has been set, then the
provided custom client object must be an instance of the client library
supported by that backend. If the backend has not been set, then the custom
client object must be an instance of a valid supported client. In this case,
sherlock
will set the backend by instrospecting the type of the provided
client object.
The global default client object set using the client parameter will be used
only by sherlock.Lock
instances. Other Backend Specific Locks
will either use the provided client object at the time of instantiating the
lock object of their types or will default to creating a simple client object
by themselves for their backend store, which will assume that their backend
store is running on localhost.
Note
this configuration cannot be overriden at the time of creating a class object.
Example:
>>> import redis
>>> import sherlock
>>>
>>> # Configure just the backend
>>> sherlock.configure(backend=sherlock.backends.REDIS)
>>>
>>> # Configure the global client object. This sets the client for all the
>>> # locks.
>>> sherlock.configure(client=redis.StrictRedis())
And when the provided client object does not match the supported library for the set backend:
>>> import etcd
>>> import sherlock
>>>
>>> sherlock.configure(backend=sherlock.backends.REDIS)
>>> sherlock.configure(client=etcd.Client())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 148, in configure
_configuration.update(**kwargs)
File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 250, in update
setattr(self, key, val)
File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 214, in client
self.backend['name']))
ValueError: Only a client of the redis library can be used when using REDIS as the backend store option.
And when the backend is not configured:
>>> import redis
>>> import sherlock
>>>
>>> # Congiure just the client, this will configure the backend to
>>> # sherlock.backends.REDIS automatically.
>>> sherlock.configure(client=redis.StrictRedis())
And when the client object passed as argument for client parameter is not a valid client object at all:
>>> import sherlock
>>> sherlock.configure(client=object())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 148, in configure
_configuration.update(**kwargs)
File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 250, in update
setattr(self, key, val)
File "/Users/vkapoor/Development/sherlock/sherlock/__init__.py", line 221, in client
raise ValueError('The provided object is not a valid client'
ValueError: The provided object is not a valid client object. Client objects can only be instances of redis library's client class, python-etcd library's client class or pylibmc library's client class.
namespace
¶
The namespace
parameter allows you to configure sherlock
to set keys
for synchronizing locks with a namespace so that if you are using the same
datastore for something else, the keys set by locks don’t conflict with your
other keys set by your application.
In case of Redis and Memcached, the name of your locks are prepended with
NAMESPACE_
(where NAMESPACE
is the namespace set by you). In case of
Etcd, a directory with the name same as the NAMESPACE
you provided is
created and the locks are created in that directory.
By default, sherlock
does not namespace the keys set for locks.
Note
this configuration can be overriden at the time of creating a class object.
expire
¶
This parameter can be used to set the expiry of locks. When set to None
,
the locks will never expire.
This parameter’s value defaults to 60 seconds
.
Note
this configuration can be overriden at the time of creating a class object.
Example:
>>> import sherlock
>>> import time
>>>
>>> # Configure locks to expire after 2 seconds
>>> sherlock.configure(expire=2)
>>>
>>> lock = sherlock.Lock('my_lock')
>>>
>>> # Acquire the lock
>>> lock.acquire()
True
>>>
>>> # Sleep for 2 seconds to let the lock expire
>>> time.sleep(2)
>>>
>>> # Acquire the lock
>>> lock.acquire()
True
timeout
¶
This parameter can be used to set after how much time should sherlock
stop trying to acquire an already acquired lock.
This parameter’s value defaults to 10 seconds
.
Note
this configuration can be overriden at the time of creating a class object.
Example:
>>> import sherlock
>>>
>>> # Configure locks to timeout after 2 seconds while trying to acquire an
>>> # already acquired lock
>>> sherlock.configure(timeout=2, expire=10)
>>>
>>> lock = sherlock.Lock('my_lock')
>>>
>>> # Acquire the lock
>>> lock.acquire()
True
>>>
>>> # Acquire the lock again and let the timeout elapse
>>> lock.acquire()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "sherlock/lock.py", line 170, in acquire
'lock.' % self.timeout)
sherlock.lock.LockTimeoutException: Timeout elapsed after 2 seconds while trying to acquiring lock.
retry_interval
¶
This parameter can be used to set after how much time should sherlock
try to acquire lock after failing to acquire it. For example, a log has already
been acquired, then when another lock object tries to acquire the same lock,
it fails. This parameter sets the time interval for which we should sleep
before retrying to acquire the lock.
This parameter can be set to 0 to continuously try acquiring the lock. But that will also mean that you are bombarding your datastore with requests one after another.
This parameter’s value defaults to 0.1 seconds (100 milliseconds)
.
Note
this configuration can be overriden at the time of creating a class object.
Generic Locks¶
sherlock
provides generic locks that can be globally configured to use a
specific backend, so that most of your application code does not have to care
about which backend you are using for lock synchronization and makes it easy
to change backend without changing a ton of code.
-
class
sherlock.
Lock
(lock_name, **kwargs)[source]¶ A general lock that inherits global coniguration and provides locks with the configured backend.
Note
to use
Lock
class, you must configure the global backend to use a particular backend. If the global backend is not set, calling any method on instances ofLock
will throw exceptions.Basic Usage:
>>> import sherlock >>> from sherlock import Lock >>> >>> sherlock.configure(sherlock.backends.REDIS) >>> >>> # Create a lock instance >>> lock = Lock('my_lock') >>> >>> # Acquire a lock in Redis running on localhost >>> lock.acquire() True >>> >>> # Check if the lock has been acquired >>> lock.locked() True >>> >>> # Release the acquired lock >>> lock.release() >>> >>> # Check if the lock has been acquired >>> lock.locked() False >>> >>> import redis >>> redis_client = redis.StrictRedis(host='X.X.X.X', port=6379, db=2) >>> sherlock.configure(client=redis_client) >>> >>> # Acquire a lock in Redis running on X.X.X.X:6379 >>> lock.acquire() >>> >>> lock.locked() True >>> >>> # Acquire a lock using the with_statement >>> with Lock('my_lock') as lock: ... # do some stuff with your acquired resource ... pass
Parameters: - lock_name (str) – name of the lock to uniquely identify the lock between processes.
- namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
- expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
- timeout (float) – set timeout to acquire lock
- retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
Note
this Lock object does not accept a custom lock backend store client object. It instead uses the global custom client object.
-
acquire
(blocking=True)¶ Acquire a lock, blocking or non-blocking.
Parameters: blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True. Returns: if the lock was successfully acquired or not Return type: bool
-
locked
()¶ Return if the lock has been acquired or not.
Returns: True indicating that a lock has been acquired ot a shared resource is locked. Return type: bool
-
release
()¶ Release a lock.
Backend Specific Locks¶
sherlock
provides backend specific Lock classes as well which can be optionally
used to use different backend than a globally configured backend. These locks
have the same interface and semantics as sherlock.Lock
.
Redis based Locks¶
-
class
sherlock.
RedisLock
(lock_name, **kwargs)[source]¶ Implementation of lock with Redis as the backend for synchronization.
Basic Usage:
>>> import redis >>> import sherlock >>> from sherlock import RedisLock >>> >>> # Global configuration of defaults >>> sherlock.configure(expire=120, timeout=20) >>> >>> # Create a lock instance >>> lock = RedisLock('my_lock') >>> >>> # Acquire a lock in Redis, global backend and client configuration need >>> # not be configured since we are using a backend specific lock. >>> lock.acquire() True >>> >>> # Check if the lock has been acquired >>> lock.locked() True >>> >>> # Release the acquired lock >>> lock.release() >>> >>> # Check if the lock has been acquired >>> lock.locked() False >>> >>> # Use this client object >>> client = redis.StrictRedis() >>> >>> # Create a lock instance with custom client object >>> lock = RedisLock('my_lock', client=client) >>> >>> # To override the defaults, just past the configurations as parameters >>> lock = RedisLock('my_lock', client=client, expire=1, timeout=5) >>> >>> # Acquire a lock using the with_statement >>> with RedisLock('my_lock') as lock: ... # do some stuff with your acquired resource ... pass
Parameters: - lock_name (str) – name of the lock to uniquely identify the lock between processes.
- namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
- expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
- timeout (float) – set timeout to acquire lock
- retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
- client – supported client object for the backend of your choice.
-
acquire
(blocking=True)¶ Acquire a lock, blocking or non-blocking.
Parameters: blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True. Returns: if the lock was successfully acquired or not Return type: bool
-
locked
()¶ Return if the lock has been acquired or not.
Returns: True indicating that a lock has been acquired ot a shared resource is locked. Return type: bool
-
release
()¶ Release a lock.
Etcd based Locks¶
-
class
sherlock.
EtcdLock
(lock_name, **kwargs)[source]¶ Implementation of lock with Etcd as the backend for synchronization.
Basic Usage:
>>> import etcd >>> import sherlock >>> from sherlock import EtcdLock >>> >>> # Global configuration of defaults >>> sherlock.configure(expire=120, timeout=20) >>> >>> # Create a lock instance >>> lock = EtcdLock('my_lock') >>> >>> # Acquire a lock in Etcd, global backend and client configuration need >>> # not be configured since we are using a backend specific lock. >>> lock.acquire() True >>> >>> # Check if the lock has been acquired >>> lock.locked() True >>> >>> # Release the acquired lock >>> lock.release() >>> >>> # Check if the lock has been acquired >>> lock.locked() False >>> >>> # Use this client object >>> client = etcd.Client() >>> >>> # Create a lock instance with custom client object >>> lock = EtcdLock('my_lock', client=client) >>> >>> # To override the defaults, just past the configurations as parameters >>> lock = EtcdLock('my_lock', client=client, expire=1, timeout=5) >>> >>> # Acquire a lock using the with_statement >>> with EtcdLock('my_lock') as lock: ... # do some stuff with your acquired resource ... pass
Parameters: - lock_name (str) – name of the lock to uniquely identify the lock between processes.
- namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
- expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
- timeout (float) – set timeout to acquire lock
- retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
- client – supported client object for the backend of your choice.
-
acquire
(blocking=True)¶ Acquire a lock, blocking or non-blocking.
Parameters: blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True. Returns: if the lock was successfully acquired or not Return type: bool
-
locked
()¶ Return if the lock has been acquired or not.
Returns: True indicating that a lock has been acquired ot a shared resource is locked. Return type: bool
-
release
()¶ Release a lock.
Memcached based Locks¶
-
class
sherlock.
MCLock
(lock_name, **kwargs)[source]¶ Implementation of lock with Memcached as the backend for synchronization.
Basic Usage:
>>> import pylibmc >>> import sherlock >>> from sherlock import MCLock >>> >>> # Global configuration of defaults >>> sherlock.configure(expire=120, timeout=20) >>> >>> # Create a lock instance >>> lock = MCLock('my_lock') >>> >>> # Acquire a lock in Memcached, global backend and client configuration >>> # need not be configured since we are using a backend specific lock. >>> lock.acquire() True >>> >>> # Check if the lock has been acquired >>> lock.locked() True >>> >>> # Release the acquired lock >>> lock.release() >>> >>> # Check if the lock has been acquired >>> lock.locked() False >>> >>> # Use this client object >>> client = pylibmc.Client(['X.X.X.X'], binary=True) >>> >>> # Create a lock instance with custom client object >>> lock = MCLock('my_lock', client=client) >>> >>> # To override the defaults, just past the configurations as parameters >>> lock = MCLock('my_lock', client=client, expire=1, timeout=5) >>> >>> # Acquire a lock using the with_statement >>> with MCLock('my_lock') as lock: ... # do some stuff with your acquired resource ... pass
Parameters: - lock_name (str) – name of the lock to uniquely identify the lock between processes.
- namespace (str) – Optional namespace to namespace lock keys for your application in order to avoid conflicts.
- expire (float) – set lock expiry time. If explicitly set to None, lock will not expire.
- timeout (float) – set timeout to acquire lock
- retry_interval (float) – set interval for trying acquiring lock after the timeout interval has elapsed.
- client – supported client object for the backend of your choice.
-
acquire
(blocking=True)¶ Acquire a lock, blocking or non-blocking.
Parameters: blocking (bool) – acquire a lock in a blocking or non-blocking fashion. Defaults to True. Returns: if the lock was successfully acquired or not Return type: bool
-
locked
()¶ Return if the lock has been acquired or not.
Returns: True indicating that a lock has been acquired ot a shared resource is locked. Return type: bool
-
release
()¶ Release a lock.