Use of service with Python

Programs have been developed on python3 in Ubuntu distribution.

To install python3 library which allows the connection to OpenCayle service run:

  • Ubuntu Installation
apt install python3-boto
  • CentOS Installation
yum install python-boto
  • MacOS High Sierra and following versions Installation

Execute on terminal: 

pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep version

when it is not installed return “ No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'. “

Install Xcode the first time:

 xcode-select --install

Check Homebrew is installed:

 which brew

When it is nothing returned Homebrew is not installed.
Install Homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Insert Homebrew folder at the beginning of PATH:

export PATH="/usr/local/opt/python/libexec/bin:$PATH"

Install python3:

brew install python3

Check Python version:

python3 --version

When returns Python 3.X.X Python3 it was installed sucessfully.
Install six library to execute OpenCayle Python scripts.

pip install six

Install boto 2 library:

pip install boto

NOTE: To download python programs (.py extension) , please, follow the instructions:
1) Select the file
2) Press the right button on your mouse
3) Select the "save as" option

Program: s3CheckConnection.py.
Parameters:
• AccessKey : Token to connect to OpenCayle service.

Execution when AccessKey is wrong:

python3 s3CheckConnection.py failedtoken
    The connection to OpenCayle service is not possible.
    Please, check the following issues to solve it before connect to Scayle:
    1) You have internet access
    2) Your internet access is not through a VPN
    3) When you generate the access key the USERID and PASSWORD were correct
    4) Your IP internet address is available on OpenCayle service

Execution when AccessKey is correct:

python3 s3CheckConnection.py XXXX  (Where XXX is the correct AccessKey for the user)
    Your AccessKey connects successfully to OpenCayle.

Program: s3CreateBucket.py.
Parameters:

  • Access Key: Token to connect to OpenCayle service.
  • BucketName: The name associated to the new bucket.

Execution when AccessKey is wrong:

python3 s3CreateBucket.py XXerrXX buckecillo

    !!! WARNING Bucket pepe can not be created. 
    Review S3ResponseError: 403 Forbidden
    <?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><RequestId>tx000000000000000296bd3-005ee33619-cbcb5-default</RequestId><HostId>cbcb5-default-default</HostId></Error> log information. 
    Please attention to field CODE if exists. 
    When CODE is AccessDenied then Connection to OpenCayle can not be done, please review 
    your AccessKey:
    1) Run program s3AccessKey.py
    2) Run program s3CheckConnection.py
    If problem persists then contact to Scayle through HelpDesk tool.

Execution when number or buckets is exceeded:

python3 s3CreateBucket.py XXXX  buckecillo
    PARAMETERS: Access Key <XXXX Bucket <buckecillo>.
    !!! WARNING Bucket buckecillo can not be created.
    Review S3ResponseError: 400 Bad Request
    <?xml version="1.0" encoding="UTF-8"?><Error><Code>TooManyBuckets</Code><BucketName>buckecillo</BucketName><RequestId>tx00000000000000000472c-005e6a087f-196c5-default</RequestId><HostId>196c5-default-default</HostId></Error> log information.
    Please attention to field CODE if exists
    When CODE is AccessDenied then Connection to OpenCayle can not be done, please review your AccessKey:
    1) Run program s3AccessKey.py
    2) Run program s3CheckConnection.py
    If problem persists then contact to Scayle through HelpDesk tool.

Successfully execution:

python3 s3CreateBucket.py XXXX buckecillo
    PARAMETERS: Access Key XXXX Bucket <buckecillo>.
    Bucket buckecillo created successfully.

Program: s3CreateKey.py.
Parameters:

  • Access Key: Token to connect to OpenCayle service.
  • BucketName: The name associated to the bucket which contains the object.
  • KeyName: The name associated to the new object of the bucket.
  • FileName: Absolute path of file associated to key.
  • Is Private 0 Public /1 Private.
  • ExpirationTime: Time on seconds to expiration URL (Private mode).

Execution when AccessKey is wrong:

python3 s3CreateKey.py XXerrXX buckecillo pepe
    Connection to OpenCayle can not be done, please review your AccessKey:
    1) Run program s3AccessKey.py
    2) Run program s3CheckConnection.py
    If problem persists then contact to Scayle through HelpDesk tool.

Execution when file does not exist:

python3 s3CreateKey.py XXXXX buckecillo pepe xx.csv 0 280
    PARAMETERS: Access Key <XXXXX> Bucket Name <buckecillo> Key Name <pepe> File Name <xx.csv> Is Private (0 Not/1 Yes) <0> ExpirationTime (0 not expiration) <280>.
    File <xx.csv> does not exist. Key pepe can be created on bucket buckecillo

Execution to assign key to a bucket which does not exit:

python3 s3CreateKey.py  XXXXX buckecillo pepe data.csv 0 280
    PARAMETERS: Access Key  <XXXXX> Bucket Name <buckecillo> Key Name <pepe> File Name <data.csv> Is Private (0 Not/1 Yes) <0> ExpirationTime (0 not expiration) <280>.
    Bucket, whose name is buckecillo, does not exist

Execution when quota is exceeded:

python3 s3CreateKey.py XXXXX buckecillo pepe /home/cephuser/data/data.csv 0 280
    PARAMETERS: Access Key <XXXXX> Bucket Name <buckecillo> Key Name <pepe> File Name </home/cephuser/data/data.csv> Is Private (0 Not/1 Yes) <0> ExpirationTime (0 not expiration) <280>.
    !!! KEY WARNING. Error during key pepe creation.  
    Error description S3ResponseError: 403 Forbidden
    <?xml version="1.0" encoding="UTF-8"?><Error><Code>QuotaExceeded</Code><BucketName>buckecillo</BucketName><RequestId>tx000000000000000004b06-005e6a1b18-196b3-default</RequestId><HostId>196b3-default-default</HostId></Error>.
    Review field CODE

Successfully creation:

python3 s3CreateKey.py XXXXX buckecillo pepe /home/cephuser/data/data.csv 0 280
    PARAMETERS: Access Key <XXXXX> Bucket Name <buckecillo> Key Name <pepe> File Name </home/cephuser/data/data.csv> Is Private (0 Not/1 Yes) <0> ExpirationTime (0 not expiration) <280>.
    Key pepe on bucket buckecillo created on mode public-read and security False. If it is secure, its expiration time is 0.
    URL associated is: https://ss3.scayle.es:443/buckecillo/pepe
    Key pepe associated to Bucket buckecillo created successfully.

Go to your browser and navigate to the URL associated:  
    https://ss3.scayle.es:443/buckecillo/pepe
    * On Explorer and Firefox browser will appear a message whick ask you about open or save the file associated to the key
    * On Chrome the browser download the file and you can access to the file associated to the key on the download folder of the machine which is using the OpenCayle service.

Program: s3DeleteKey.py.
Parameters:

  • AccessKey : Token to connect to OpenCayle service
  • BucketName: The name associated to the bucket which contains the object
  • KeyName: The name associated to the object to delete

Execution when AccessKey is wrong:

python3 s3DeleteKeyt.py XXerrXX  buckecillo dolores
    Connection to OpenCayle can not be done, please review your AccessKey:
    1) Run program s3AccessKey.py
    2) Run program s3CheckConnection.py
    If problem persists then contact to Scayle through HelpDesk tool.

Execution when the bucket does not exist:

python3 s3DeleteKey.py XXXX  buckecillo pepe
    PARAMETERS: Access Key <XXXX> Bucket Name <buck> Key Name <pepe>.
    Bucket buckecillo or Key pepe associated to this bucket does not exist. Please review your S3 data.

Execution when the key does not exist but the bucket exist:

python3 s3DeleteKey.py XXXX  buckecillo dolores
    PARAMETERS: Access Key <XXXX> Bucket Name <buckecillo> Key Name <dolores>.
    Bucket buckecillo or Key dolores associated to this bucket does not exist. Please review your S3 data.

Successfully key deletion:

python3 s3DeleteKey.py XXXX  buckecillo pepe
    PARAMETERS: Access Key <XXXX> Bucket Name <buckecillo> Key Name <pepe>.
    Key pepe deleted from bucket buckecillo successfully

Program: s3DeleteBucket.py.
Parameters:

  • AccessKey : Token to connect to OpenCayle service.
  • BucketName: The name associated to the bucket which contains the object.
  • KeyName: The name associated to the object to delete.

Execution when AccessKey is wrong:

python3 s3DeleteKeyt.py XXerrXX  buckecillo dolores
    Connection to OpenCayle can not be done, please review your AccessKey:
    1) Run program s3AccessKey.py
    2) Run program s3CheckConnection.py
    If problem persists then contact to Scayle through HelpDesk tool.

Execution when bucket does not exist:

python3 s3DeleteBucket.py XXXXX  buckell
    PARAMETERS: Access Key  XXXXX Bucket <buckell>.
    Bucket buckecillo does not exist. Please review your S3 data

Execution when bucket exists and have keys associated:

Step 1) Create a new key associated to the bucket.

python3 s3CreateKey.py XXXXX buckecillo lola /home/cephuser/data/data.csv 0 280
    PARAMETERS: Access Key <XXXXX> Bucket Name <buckecillo> Key Name <pepe> File Name </home/cephuser/data/data.csv> Is Private (0 Not/1 Yes) <0> ExpirationTime (0 not expiration) <280>.
    Key pepe on bucket buckecillo created on mode public-read and security False. If it is secure, its expiration time is 0.
    URL associated is: https://ss3.scayle.es:443/buckecillo/lola
    Key lola associated to Bucket buckecillo created successfully.

Step 2) Delete bucket which contains the key created on step 1.

python3 s3DeleteBucket.py XXXXX buckecillo
    PARAMETERS: Access Key <XXXXX> Bucket <buckecillo>.
    Bucket buckecillo deleted successfully. All their keys were deleted previously

Program: s3Resources.py.
Parameters:

  • AccessKey : Token to connect to OpenCayle service.

Execution when AccessKey is wrong :

python3 s3Resources.py  XX
    PARAMETERS: Access Key <XX>
-----------------------------------------------------------------------------------------
                         Open Scayle Resources associated to user.  
-----------------------------------------------------------------------------------------
    Connection to OpenCayle can not be done, please review your AccessKey:
    1) Run program s3AccessKey.py
    2) Run program s3CheckConnection.py
    If problem persists then contact to Scayle through HelpDesk tool.

Show information when user connection information is wrong:

python3 s3Resources.py  XX
    PARAMETERS: Access Key <XX>
    boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
    <?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><RequestId>tx000000000000000004fbf-005e6a2b79-196b3-default</RequestId><HostId>196b3-default-default</HostId></Error>

Show information when user connection information is correct:

python3 s3Resources.py  XXXX
    PARAMETERS: Access Key <XXXX>
-----------------------------------------------------------------------------------------
                         OpenCayle Resources associated to user.  
-----------------------------------------------------------------------------------------
    - BUCKET bucket1 2020-03-12T12:22:39.216Z
      - OBJECT/KEY alejandra        2020-03-12T12:24:01.011Z
      - OBJECT/KEY alex     2020-03-12T12:23:53.648Z
    - BUCKET bucket2 2020-03-12T12:22:42.362Z
      - OBJECT/KEY juan     2020-03-12T12:23:28.295Z
      - OBJECT/KEY lola     2020-03-12T12:23:43.375Z
-----------------------------------------------------------------------------------------
                         END OpenCayle Resources