{"id":35455,"date":"2023-05-17T12:27:52","date_gmt":"2023-05-17T12:27:52","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=35455"},"modified":"2023-05-17T12:34:10","modified_gmt":"2023-05-17T12:34:10","slug":"user-data-commands-run-in-each-ec2-instance-in-sequential-not-parallel","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/user-data-commands-run-in-each-ec2-instance-in-sequential-not-parallel\/","title":{"rendered":"Aws: user data commands run in each ec2 instance in sequential not parallel"},"content":{"rendered":"\n

AWS Approach: aws ec2 instance user data to make sure commands run in each ec2 instance in sequential not parallel<\/p>\n\n\n\n

Approach # 1<\/strong><\/h2>\n\n\n\n

To ensure that commands specified in the EC2 instance user data are executed sequentially rather than in parallel on multiple instances, you can use a synchronization mechanism such as a lock file or a distributed coordination service. Here’s an approach using a lock file:<\/p>\n\n\n\n

    \n
  1. Create a lock file in a shared location that is accessible by all EC2 instances. For example, you can use Amazon S3 or a shared network file system (NFS) mounted on each instance.<\/li>\n\n\n\n
  2. Modify the EC2 instance user data script to check for the existence of the lock file before executing any commands. If the lock file exists, the script waits until the lock file is released; otherwise, it creates the lock file and continues with the execution.<\/li>\n<\/ol>\n\n\n\n

    Here’s an example of how you can modify the user data script:<\/p>\n\n\n\n

    \n\n\n
    #!\/bin\/bash<\/span>\r\nlockfile=\"\/path\/to\/lockfile\"<\/span>\r\n\r\n# Check if the lock file exists<\/span>\r\nwhile<\/span> [ -f \"$lockfile\"<\/span> ]; do<\/span>\r\n    echo<\/span> \"Lock file exists. Waiting for lock release...\"<\/span>\r\n    sleep 10<\/span>\r\ndone\r\n\r\n# Create the lock file<\/span>\r\ntouch \"$lockfile\"<\/span>\r\n\r\n# Execute your sequential commands here<\/span>\r\necho<\/span> \"Executing command 1...\"<\/span>\r\n<command 1<\/span>>\r\necho<\/span> \"Executing command 2...\"<\/span>\r\n<command 2<\/span>>\r\n\r\n# Release the lock file<\/span>\r\nrm \"$lockfile\"<\/span>\r\n<\/code><\/span>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
    \n\n\n
    #!\/bin\/bash<\/span>\r\nbucket_name=\"your-s3-bucket-name\"<\/span>\r\nlockfile_key=\"lockfile\"<\/span>\r\n\r\n# Check if the lock file exists in S3 bucket<\/span>\r\nwhile<\/span> aws s3 ls \"s3:\/\/$bucket_name\/$lockfile_key\"<\/span> >\/dev\/null<\/span> 2<\/span>>&1<\/span>; do<\/span>\r\n    echo<\/span> \"Lock file exists. Waiting for lock release...\"<\/span>\r\n    sleep 10<\/span>\r\ndone\r\n\r\n# Create the lock file<\/span>\r\naws s3 cp \/dev\/null<\/span> \"s3:\/\/$bucket_name\/$lockfile_key\"<\/span>\r\n\r\n# Execute your sequential commands here<\/span>\r\necho<\/span> \"Executing command 1...\"<\/span>\r\n<command 1<\/span>>\r\necho<\/span> \"Executing command 2...\"<\/span>\r\n<command 2<\/span>>\r\n\r\n# Release the lock file<\/span>\r\naws s3 rm \"s3:\/\/$bucket_name\/$lockfile_key\"<\/span>\r\n<\/code><\/span>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
    \n\n\n
    #!\/bin\/bash<\/span>\r\nbucket_name=\"your-s3-bucket-name\"<\/span>\r\nlockfile_key=\"lockfile\"<\/span>\r\n\r\n# Check if the lock file exists in S3 bucket<\/span>\r\nwhile<\/span> aws s3api head-object --bucket \"$bucket_name\"<\/span> --key \"$lockfile_key\"<\/span> >\/dev\/null<\/span> 2<\/span>>&1<\/span>; do<\/span>\r\n    echo<\/span> \"Lock file exists. Waiting for lock release...\"<\/span>\r\n    sleep 10<\/span>\r\ndone\r\n\r\n# Create the lock file<\/span>\r\naws s3api put-object --bucket \"$bucket_name\"<\/span> --key \"$lockfile_key\"<\/span> --body \/dev\/null<\/span>\r\n\r\n# Execute your sequential commands here<\/span>\r\necho<\/span> \"Executing command 1...\"<\/span>\r\n<command 1<\/span>>\r\necho<\/span> \"Executing command 2...\"<\/span>\r\n<command 2<\/span>>\r\n\r\n# Release the lock file<\/span>\r\naws s3api delete-object --bucket \"$bucket_name\"<\/span> --key \"$lockfile_key\"<\/span>\r\n<\/code><\/span>Code language:<\/span> PHP<\/span> (<\/span>php<\/span>)<\/span><\/small><\/pre>\n\n\n
    \n\n\n\n
    Approach # <\/strong>2<\/h2>\n\n\n\n
    \"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"
    AWS Approach: aws ec2 instance user data to make sure commands run in each ec2 instance in sequential not parallel Approach # 1 To ensure that commands specified in the EC2 instance user data are executed sequentially rather than in parallel on multiple instances, you can use a synchronization mechanism such as a lock file…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_joinchat":[],"footnotes":""},"categories":[2],"tags":[],"class_list":["post-35455","post","type-post","status-publish","format-standard","hentry","category-uncategorised"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/35455","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=35455"}],"version-history":[{"count":4,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/35455\/revisions"}],"predecessor-version":[{"id":35471,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/35455\/revisions\/35471"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=35455"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=35455"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=35455"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}