Apache HTTP Server module for storing and reading BLOBs to and from a cloud service.
Storing and reading BLOBs to and from a cloud service. The project will consist of a library, command line tool, Apache HTTP Server module and an Nginx module. Currently, the command line tool is work in progress.
See Releases
The command line tool is controlled via env variables and command line arguments. Command line arguments take priority and overwrite env variables settings. If the action is set to WRITE_BLOB and neither MCS_PATH_TO_FILE nor –path_to_file is specified, the command line tool tries to read from stdin. Similarly, when then action is set to READ_BLOB, and neither MCS_PATH_TO_FILE nor –path_to_file is specified, the command line tool writes the blob contents to stdout.
MCS_ACTION=READ_BLOB|WRITE_BLOB|DELETE_BLOB|LIST_BLOBS|CREATE_CONTAINER
MCS_AZURE_STORAGE_KEY=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
MCS_AZURE_STORAGE_ACCOUNT=devstoreaccount1
MCS_AZURE_BLOB_NAME=test
MCS_AZURE_CONTAINER=your-container
MCS_PATH_TO_FILE=/tmp/meh
MCS_BLOB_STORE_URL=127.0.0.1:10000
MCS_TEST_REGIME=true|false
--action
--azure_storage_key
--azure_storage_account
--blob_name
--azure_container
--path_to_file
--blob_store_url
--test_regime
./mod_cloud_storage \
--action CREATE_CONTAINER \
--azure_storage_key Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw== \
--azure_storage_account devstoreaccount1 \
--azure_container my-first-container \
--blob_store_url 127.0.0.1:10000 \
--test_regime true
Note the storage account is always called devstoreaccount1 by definition in all Azure Storage emulator implementations. The same holds for the storage key. Container and blob names are arbitrary.
echo "Silence is golden." > /tmp/test
./mod_cloud_storage \
--action WRITE_BLOB \
--azure_storage_key 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==' \
--azure_storage_account 'devstoreaccount1' \
--azure_container mod-cloud-storage \
--blob_store_url localhost:10000 \
--test_regime true \
--blob_name 'testblob' < /tmp/test
Output (DEBUG):
Reading from stdin
BLOB to be uploaded length: 19
BLOB to be uploaded MD5sum base64: YmT6qz9/oJawL1eym/1EQQ==
BLOB to be uploaded MD5sum hex: 6264faab3f7fa096b02f57b29bfd4441
libcurl said: No error
Response Code: 201
Response Body followed by \n:(null)
Note the tool is reading from stdin via pipe now …< /tmp/test. One could use either MCS_PATH_TO_FILE env var or –path_to_file parameter instead. If all aforementioned is specified, parameter takes precedence over env var and env var takes precedence over stdin.
./mod_cloud_storage \
--action READ_BLOB \
--azure_storage_key 'Eby8vdM02xNOcqFlqUwJPLlm\EtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/\K1SZFPTOtr/KBHBeksoGMGw==' \
--azure_storage_account 'devstoreaccount1' \
--azure_container mod-cloud-storage \
--blob_store_url localhost:10000 \
--test_regime true \
--blob_name 'testblob'
Neither MCS_PATH_TO_FILE nor –path_to_file was specified, so the content of the BLOB was written to stdout. Output (DEBUG):
libcurl said: No error
Response Code: 200
Response Body followed by \n:Silence is golden.
Note that if a path to the file is specified, the content of the file is overwritten, not appended to.
Exactly the same as reading the BLOB but for the action
parameter:
--action DELETE_BLOB
Output (DEBUG):
libcurl said: No error
Response Code: 202
Response Body followed by \n:(null)
Note that the current implementation deletes all blob’s snapshots too. Regarding leases, the current implementation does not deal with leases on blobs, so if there is a lease on the blob, the delete action fails.
./mod_cloud_storage \
--action CREATE_CONTAINER \
--azure_storage_key 'YourStorageKey' \
--azure_storage_account karmdelete \
--azure_container my-first-container
echo "Silence is golden." > /tmp/test
./mod_cloud_storage \
--action WRITE_BLOB \
--azure_storage_key 'YourStorageKey' \
--azure_storage_account karmdelete \
--azure_container my-first-container \
--blob_name 'testblob' \
--path_to_file /tmp/test
Output (DEBUG):
Reading from file.
BLOB to be uploaded length: 19
BLOB to be uploaded MD5sum base64: YmT6qz9/oJawL1eym/1EQQ==
BLOB to be uploaded MD5sum hex: 6264faab3f7fa096b02f57b29bfd4441
libcurl said: No error
Response Code: 201
Response Body followed by \n:(null)
./mod_cloud_storage \
--action READ_BLOB \
--azure_storage_key 'YourStorageKey' \
--azure_storage_account karmdelete \
--azure_container my-first-container \
--blob_name 'testblob' \
--path_to_file /tmp/test_out
Output (DEBUG):
libcurl said: No error
Response Code: 200
Writing to file: /tmp/test_out
19 bytes of response written to /tmp/test_out.
DELETE_BLOB
.This project is maintained by Karm
Hosted on GitHub Pages — Theme by orderedlist