AWS SAP Notes 08 - App Services, Containers and Serverless
aws sap
15
AWS
44
White

Nguyễn Huy Hoàng viết ngày 10/10/2021

ECS - Elastic Container Service

  • It is a service that accepts containers and orchestrates where and how to run those containers
  • It is a managed container based compute service
  • It runs on two modes: EC2 and Fargate
  • Cluster: is the place where container run based on how we want them to run
  • Containers are located in container registries (ECR, DockerHub)
  • ECS uses container definitions to locate images in the container registries, which port should the image use, etc. providing information about the container we want to run
  • Task definition: represents a self-contained application, can have one or many containers defined in it. A task in ECS defines the application as a whole
  • Task definitions store the resources to be used (CPU, memory), networking configuration, compatibility (EC2 mode or Fargate) and also they store the task role (IAM role)
  • Task role give permission to ECS containers to access AWS services
  • A task does not scale by its own and it is not HA
  • ECS service: it is configured by a service definition. In a service we define how we want a task to scale, how many copies we like to run
  • ECS services define scalability and HA for tasks

ECS Cluster Modes

  • Cluster mode define how much of the admin overhead is required for running containers in ECS (what parts do we manage and what parts does AWS manage)
  • Cluster modes are:
    • EC2 Mode
    • Fargate Mode

EC2 Mode

  • Uses EC2 instances which are running inside of a VPC
  • Since we are inside of a VPC, we can benefit from using multiple AZs
  • When we create the cluster, we specify the initial size of containers
  • Horizontal scaling for EC2 instances and for ECS tasks is controlled by ASGs
  • With EC2 cluster mode we are paying for EC2 instances independently of what containers and how many of containers are running on them

Fargate Mode

  • We don't have to manage EC2 instances for use as container host
  • With Fargate there are no servers to manage
  • AWS maintains a shared Farge infrastructure platform offered to all users
  • We gain access to resources from a shared pool, we don't have visibility for other customers
  • A Fargate deployment still uses a cluster with a VPC which operates in AZs
  • ECS tasks are injected into the VPC with an ENI, they are running on the Fargate shared platform
  • With Fargate mode we only pay for the containers we using based on the resources they consume

EC2 vs ECS (EC2) vs Fargate

  • If we are already using containers, we should use ECS
  • Containers make sense if we want to isolate applications
  • We generally pick EC2 mode if we have a large workload and the business is price conscious
  • Historically EC2 mode was giving the most value for the price if we were using saving plans. Nowadays we can have savings plan for Fargate and Lambda, so we should default ot Fargate instead of EC2 mode
  • If we are overhead conscious, we should use Fargate
  • For small/burst workloads we should use Fargate as well. Same is recommended for batch/periodic workloads

SNS - Simple Notification Service

  • It is a highly available, durable, secure, pub-sub messaging service
  • It is a public AWS service: requires network connectivity with a public endpoint
  • It coordinates the sending and delivery of messages
  • Messages are <= 256 KB payloads
  • SNS Topics are the base entity of SNS. On this topics are permissions controlled and configurations defined
  • A publisher sends messages to a topic
  • Topics can have subscribers which will receive messages from a topic
  • Supported subscribers are:
    • HTTP(s)
    • Email (JSON)
    • SQS queues
    • Mobile Push Notifications
    • SMS messages
    • Lambda Functions
  • SNS is used across AWS for notifications, example CloudWatch uses it extensively
  • It is possible to apply filters to a subscriber
  • Fan-out architecture: single topic with multiple SQS queue subscribers
  • SNS offers delivery status for supported subscribers (HTTP, Lambda SQS)
  • SNS supports delivery retries
  • SNS it is a HA and scalable service within a region
  • SNS supports Server Side Encryption (SSE)
  • We can use cross-account access to a topic via topic policies

SQS - Simple Queue Service

  • SQS provides managed message queues
  • It is a public AWS service
  • It is fully managed with highly available and highly performant queues by design
  • Queues come in two types: Standard (ordering is best effort) and FIFO (guarantee an order)
  • Messages added to the queue are up to 256 KB in size
  • VisibilityTimeout: when a client receives a message, the message is hidden for a period of time. The VisibilityTimeout is the period which is given to the client in order to process the message
  • If the client processes the message it has to delete the message from the queue. If the message is not deleted, after the VisibilityTimeout the message will be available for other clients
  • Dead-Letter queues: all unprocessed messages can be sent here after a given number of retries. Allows different types of processing on the messages
  • Queues can be used for scaling by ASGs, Lambdas can be invoked based on messages in the queue
  • With SQS we are billed per request: 1 request can receive 1 - 10 messages up to 64KB total
  • Polling SQS queues:
    • Short polling (immediate): can return 0 or more messages. It will return immediately if there are no messages on the queue
    • Long polling: we can specify waitTimeSecond value which can be up to 20 seconds to wait form messages when polling. It uses fewer requests than short polling
  • SQS supports encryption at rest (KMS) and encryption in transit

SQS Standard vs FIFO Queues

  • Standard:
    • Messages are delivered at least once
    • The order of the delivery is not guaranteed (best-effort ordering)
    • They don't have performance limitations
  • FIFO:
    • Messages are delivered exactly once
    • The order of the delivery is guaranteed to be the same as the messages were sent
    • Performance is limited: 3000 messages per second with batching or up to 300 messages per second without batching
    • There is also available a high throughput mode for FIFO queues
    • FIFO queues have to have .fifo suffix in order to be valid FIFO queues

SQS Extended Client Library

  • SQS has a message size limit of 256KB
  • Extended Client Library can be used when we want to send messages larger than this size
  • It can process large payloads and have the bulk of the payloads stored in S3
  • When we send a message using SendMessage API, the library uploads the content to S3 and stores a link to this content in the message
  • When receiving a message, the library loads the payload from S3 and replaces the link with the payload in the message
  • When deleting a message from a queue, the large S3 payload will also be deleted
  • The Extended Client Library has an implementation in Java

SQS Delay Queues

  • Delay queues allow us to postpone the delivery of messages in SQS queues
  • For a delay queue we configure a DelaySeconds value. Messages added to the queue will be invisible for the amount of DelaySeconds
  • The default value of DelaySeconds is 0, the max value is 15 minutes. In order for a queue to be delay queue, the value should be set to be greater than 0
  • Message times allow a per-message basis invisibility to be set, overriding the queue setting. Min/max values are the same. Per message setting is not supported for FIFO queues

SQS Dead Letter Queues (DLQ)

  • Dead Letter Queues are designed to help handle reoccurring failures while processing messages from a queue
  • Every time a message is received, the ReceiveCount is incremented. In order to not process the same message over and over again, we define a maxReceiveCount and a DLQ for a queue. After the number of retries is greater than the maxReceiveCount, the message is pushed into the DLQ
  • Having a DLQ, we can define alarms to get notified in case of a failure
  • Messages in DLQ can be further analyzed
  • Every queue (including DLQ) have a retention period for messages. Every message has an mq-timestamp which represents when was the message was added to the queue. In case a message is moved into a DLQ, this timestamp is not altered
  • Generally the retention period for a DLQ should be longer compared to normal queues
  • A single DLQ can be used for multiple sources

Amazon MQ

alt text

  • SNS and SQS are AWS services using AWS APIs
  • SNS provides topics which are one to many communication channels, SQS provides queues which are one to one communication channels
  • Both SNS and SQS are public services, HA and AWS integrated
  • Larger organization might already use on-premise messaging systems, which are not entirely compatible with SNS and SQS
  • Amazon MQ is an open-source message broker, is a managed implementation of Apache ActiveMQ
  • It supports the JMS API and protocols such as AMQP, MQTT, OpenWire and STOMP
  • It provides both queues and topics
  • It uses message broker services which can be single instance (test, dev) and HA pair (Active/StandBy for production)
  • Unlike SQS and SNS, Amazon MQ is not a public service, it runs in a VPC
  • It does not have native integration with other AWS services in the same way as SNS/SQS
  • Amazon MQ considerations:
    • By default we should chose SNS/SQS for newer implementation
    • We should use Amazon MQ if we migrate from an existing system with little to no application change
    • We should use Amazon MQ if we need APIs such as JMS or protocols such as AMQP, MQTT, OpenWire, STOMP
    • Amazon MQ requires the appropriate private networking

AWS Lambda

  • Lambda is a Function-as-a-Service (FaaS) product. We provide specialized short running focused code for Lambda and it will take care running it and billing us for only what we consume
  • Every Lambda function uses a supported runtime, example: Python 3.8, Java 8, NodeJS
  • Every Lambda function is loaded into an executed in a runtime environment
  • When we create a function we define the resources the function will use. We define the memory directly and CPU usage allocation indirectly (based on the amount of memory)
  • We are only billed for the duration the function is running based on the number of invocations and the resources specified
  • Lambda is a key part of serverless architectures in AWS
  • Lambda has support for the following runtimes:
    • Python
    • Ruby
    • Go
    • Java
    • C#
    • Custom using Lambda layers
  • Lambda functions are stateless, meaning there is no data left over after an invocation
  • When creating a Lambda function we define the memory. The memory can be between 128MB and 3GB in 64MB steps(historically, nowadays there is support for up to 10GB)
  • We do not directly define the vCPU allocated to each function, it will automatically scale with the memory: 1792MB of memory gives 1 vCPU
  • The runtime env. has a 512MB storage available as /tmp
  • Lambda function can run up to 15 minutes, after this a timeout will occur alt text

Lambda Networking

  • Lambda functions can have 2 types of networking modes:
    • Public (default): alt text
      • Lambda can access public AWS services such as SQS, DynamoDB, etc. and also internet based services
      • Lambda has network connectivity to public services running on the internet
      • Offers the best performance for Lambda, no customer specific networking is required
      • With public networking mode Lambda function wont be able to access resources in a VPC unless the resources do have public IPs and security controls allow external access
    • VPC Networking: alt text alt text
      • Lambda functions will run inside a VPC, so they will access everything in a VPC, assuming NACLs and SGs allow access
      • They wont be able to access services outside of the VPC, unless networking configuration exists in the VPC to allow external access
      • At the creation of the function, a certain ENI might be created for accessing the VPC. The initial setup would take up to 90 seconds

Lambda Security

alt text

  • There are 2 key parts of the security model
    • Lambda Functions will assume an execution role in order to access other AWS resources
    • Resources policies: similar to resource policies for S3. Allows external accounts to invoke a Lambda functions, or certain services to use Lambda functions. Resources polices can be modified using the CLI (not with the console)

Lambda Logging

  • Lambda uses CloudWatch Logs and X-Ray
  • Logs from Lambda executions are stored in CloudWatch Logs
  • Details about Lambda metrics are stored in CloudWatch Metrics
  • Lambda can be integrated with X-Ray for distributed tracing
  • For Lambda to be able to log we need to give an execution log to it

Lambda Invocations

  • There are 3 ways Lambda functions can be invoked:
    • Synchronous invocation: alt text
      • Command line or API directly invoking the function
      • The CLI or API will wait until the function returns
      • API Gateway will also invoke Lambdas synchronously, use case for many serverless applications
      • Any errors or retries have to be handled on the client side
    • Asynchronous invocation: alt text
      • Used typically when AWS services invoke the function (example: S3 events)
      • The service will not wait for the response (fire and forget)
      • Lambda is responsible for any failure. Reprocessing will happen between 0 and 2 times
      • The function should be idempotent in order to be rerun
      • Lambda can be configured to send events to a DLQ in case of the processing did not succeed after the number of retries
      • Destination: events processed by Lambdas can be delivered to destinations like SQS, SNS, other Lambda, EventBride. Success and failure events can be sent to different destinations
    • Event Source mapping: alt text
      • Typically used on streams or queues which don't generate events (Kinesis, DynamoDB streams, SQS)
      • Event Source mappers polls these streams and retrieves batches. These batches can be broken in pieces and sent to multiple Lambda invocations for processing
      • We can not have a partially successful batch, either everything works or nothing works
  • In case of event processing in async invocation, in order to process the event we don't explicitly need rights to read from the sender
  • In case of event source mapping the event source mapper is reading from the source. The event source mapping uses permissions from the Lambda execution role to access the source service
  • Even if the function does not read data directly from the stream, the execution role needs read rights in order to handle the event batch
  • Any batch that consistently fails to be processed, it can be sent to an SQS queue or SNS topic for further processing

Lambda Versions

  • We can define different versions for given functions
  • A version of a function is the code + configuration of the function
  • When we publish a version, it becomes immutable, it no longer can be changed. It event gets its own ARN (Amazon Resource Name)
  • $Latest points to the latest version of Lambda version (it is not immutable)
  • We can also define aliases (DEV, STAGE, PROD) which point to a version of the function. Aliases can be changed to point to other versions

Lambda Start-up Times

  • Lambda code runs inside of a runtime environment (execution context)
  • At first invocation this execution context needs to be created and this will take time
  • This process is known as cold start and it can take 100ms or more
  • If the function is invoked again without too much of a gap, it might use the same execution context. This is called warm start
  • One function invocation runs in an execution environment at a time. If multiple parallel instances are needed, the contexts will require cold starts
  • Provisioned concurrency: we provision execution context in advance for Lambda invocations
  • The improve performance we can use the /tmp folder to download data to it. If another invocations uses the same execution context, it will be able to access the previously downloaded data
  • We can create database connections outside of the Lambda handler. These will also be available for other invocations afterwards

Lambda Handler

  • Lambda function executions have lifecycles
  • The function code runs inside an execution environment
  • Lifecycle phases:
    • INIT: creates on unfreezes the execution environment
    • INVOKE: runs the function handler (cold start)
    • NEXT INVOKE(s): warm start using the same environment
    • SHUTDOWN: execution environment is terminated

Lambda Versions

  • Unpublished functions can be changed and deployed
  • $LATEST version of the Lambda code can be edited and deployed
  • We can take the current state of the function and publish it which will create an immutable version
  • If the function is published, the code, dependencies, runtime settings and env. variables in the version created can not be edited
  • Each version gets an uniq ARN (Qualified ARN)
  • Unqualified ARN points at the function without a specific version ($LATEST)

Lambda Aliases

  • An alias is a pointer to a function version
  • Example: PROD => function:1, BETA => function:2
  • Each alias has an unique ARN
  • Aliases can be updated, changing which version they reference
  • Useful for PROD/DEV, BLUE/GREEN deployments, A/B testing
  • We can also use alias routing: sending a certain percentage of request to v1 and other percentage to v2. Both versions need the same role, same DLQ will be used and both versions need to be published

Lambda Environment Variables

  • Key and value pairs for associated with Lambda functions
  • By default they are associated with $LATEST - can be edited
  • If they are publishes, they can not be edited
  • They can be accessed within the execution environment
  • The environment variables can be encrypted with KMS
  • They allow code execution to be adjusted based on variables

Lambda Layers

  • Used to split off libraries and dependencies from Lambda functions
  • Reduces the size of the deployment package
  • Layers can be reused by multiple Lambda functions
  • Libraries in layers are extracted in the /opt folder
  • Layers allow new runtimes which are not explicitly supported by AWS

Lambda Container Images

  • Until recently Lambda was considered to be a Function as a Service (FaaS) product, which means creation of a function, uploading code and executing it
  • Many organizations use containers and CI/CD processes built for containers
  • Lambda is now capable to use containers images
  • It is an alternative way of packaging the function code and using it with the Lambda product
  • Lambda Runtime API - has to be included with the container images
  • AWS Lambda Runtime Interface Emulator (RIE): used for local Lambda testing

Lambda and ALB

  • Multi-Value headers:
    • Example: `http://function.io?&search=roffle&search=winkie
    • Without multi-value headers the Lambda receives the following: "queryStringParameters": { "search": "winkie" }
    • If the multi-value headers are enabled, we get this delivered to Lambda: "multiValueQueryStringParameters": { "search": ["roffle", "winkie"] }

AWS SAM - Serverless Application Model

alt text

  • Serverless application is not just Lambda function, it can include many more services such as:
    • Front end code and assets hosted by S3 and CloudFront
    • API endpoint - API Gateway
    • Compute - Lambda
    • Database - DynamoDB
    • Event sources, permissions and more...
  • SAM group of products has 2 main parts:
    • AWS SAM template specification, which is an extension of CloudFormation
    • AWS SAM CLI allowing local testing, local invocation and deployments into AWS

CloudWatch Events and EventBridge

  • Deliver a near real-time stream of system events
  • These events describe changes in AWS services, example: EC2 instance is started
  • EventBridge is a newer system replacing CloudWatch Events. It can perform the same functionality, in addition it can handle events from third-parties and custom applications
  • Both of the services operate using an event bus. Both have a default event bus
  • In CloudWatch Events there is only the default event bus, which is explicit and it is not exposed to the UI
  • In EventBridge we can have additional event buses
  • In both systems we create rules matching incoming events, or we have scheduled based rules
  • Events themselves are JSON objects, including for example which EC2 instance changed state, in what state changed into as well as other things such as date and time

API Gateway

  • Is a service which lets us create and manage APIs
  • API Gateway acts as endpoint or entry-point applications which want to talk with our services
  • Sits between the application and integrations (services)
  • API Gateway is HA and scalable
  • Handles authorization, throttling, caching, CORS, transformations
  • It also supports the OpenAPI spec and direct integration with other AWS services
  • API gateway is a public service
  • It can provide APIs using REST and WebSocket
  • API Gateway overview: alt text

API Gateway - Authentication

alt text

  • API Gateway supports a range of authentication types, example Cognito, Lambda based authentication (Custom based authentication - Bearer token), IAM credentials, etc.

API Gateway - Endpoint Types

  • Edge-Optimized: any incoming request is routed to the nearest CloudFront POP (point of presence)
  • Region: region endpoint for clients in the same region
  • Private: endpoints only accessible in a VPC via interface endpoints

API Gateway - Stages

alt text

  • When we deploy an API configuration, we are doing it into a stage
  • Example we can have prod/dev stage with uniq settings and urls
  • Deployments can be rolled back on a stage
  • On stages we can enable canary deployments. When enabled, the deployment will be made to the canary not the stage itself
  • Traffic distribution can be altered between canary and base stage
  • Canary stage can be promoted to base

API Gateway - Errors

  • 4XX - Client errors: invalid request on the client side
  • 5XX - Server errors: valid request, backend issue
  • 400 - Bad Request: generic client side error
  • 403 - Access Denied: authorizer denies request, request is WAF filtered
  • 429 - API Gateway can throttle: this means we have exceeded a specified amount of requests
  • 502 - Bad Gateway Exception: bad output returned by Lambda
  • 503 - Service Unavailable: backing endpoint is offline
  • 504 - Integration Failure/Timeout (29s limit)

API Gateway - Caching

alt text

  • Caching is configured per stage
  • We can define a cache on a stage (500 MB up to 237 GB)
  • Cache TTL default value is 300 seconds, configurable between 0 and 3600s. Can be encrypted
  • Calls only will reach the backend in case of a cache miss

API Gateway Methods and Resources

  • API Gateway URL example: https://1nj7i16t37.execute-api.us-east-1.amazonaws.com/dev/listcats
  • The URL can be represents the following: [api-gateway-endpoint]/[stage]/[resource]
  • Stages are logical configurations. APIs are deployed into stages. Stages can be used for different application versions or lifecycle points for an API
  • API changes only take effect after it is deployed into a stage
  • Methods are the desired action to be performed. Methods are HTTP verbs
  • Methods are where integrations are configured which provide the functionality of an API. Methods can integrate with Lambda, HTTP and other AWS services

API Gateway Integrations

  • API Gateway is capable of connecting to Lambda, HTTP Endpoints (running on-premises or on AWS), Step Functions, SNS, DynamoDB
  • APIs have 3 phases:
    • Request: authorize, validate and transform the request
    • Integrations
    • Response: transform, prepare and return the response
  • The request and response phases are split into 2 parts:
    • Method Request: defines everything about the client request to method
    • Integration Request: parameters from the method request are transferred to the integrations
    • Integration Response: converts the data from the backend to a form which can be sent back to the client
    • Method Response: how the communication is delivered back to the client
  • API methods which are on the client side decide what the client request to method is like. There are integrated to a backend endpoint via integrations
  • Integration types:
    • Mock: used for testing, no backed involved. No real backend needed
    • HTTP: http custom integration. We have to configure both integration request and integration response
    • HTTP Proxy: subtype of HTTP where. Allows the access HTTP endpoint with a streamline integration. Proxying is where the request is passed to the endpoint as is and sent back to the client as is
    • AWS: allows an API to expose AWS services. We have to configure both the integration request and response
    • AWS_PROXY (LAMBDA): integration request/response does not have to be defined, API Gateway passes the request unmodified
  • Mapping template: used for non-proxy integrations. Used for:
    • Modify or rename parameters
    • Modify the body or header of the request
    • Filtering - remove anything from the request
  • Mapping uses VTL (Velocity Template Langue) for editing the request

API Gateway Stages and Deployments

  • Editing an API, we are editing settings which are not live (not published)
  • The current state of the API needs to be deployed to a stage
  • Each stage has its own configuration. Configurations are not immutable, can be modified, overwritten or rolled back
  • Stage variables: environment variables for stages

Swagger and OpenAPI

  • OpenAPI (OAS) defines a standard language-agnostic interface to RESTful APIs
  • OpenAPI v2 is formerly known as Swagger
  • OpenAPI v3 is a more recent version
  • OpenAPI defines endpoints, operation (GET, POST, etc.), input and output parameters and authentication methods
  • API Gateway is capable of import OpenAPI format and generating it. Useful for backups and migrations

AWS Step Functions

  • Lambda is FaaS
  • A function has an execution time of maximum 15 minutes
  • Lambda functions can be chained together, but it is considered to be an anti-patterns and it can get messy. Lambda runtime environments are stateless
  • Step Functions is service which lets us create state machines
  • States can do things, can decide things, can take in data, modify data and output data
  • Maximum duration for state machine execution is 1 year
  • Step Functions can represent 2 different type of workflow:
    • Standard workflow: is the default and it has 1 year execution limit
    • Express workflow: designed for high volume event processing, which can run up to 5 minutes
  • Step Functions can be initiated with API Gateway, IOT Rules, EventBridge, Lambda, etc.
  • State machines for Step Functions can be created with Amazon State Language (ASL) JSON templates

States

  • Available states:
    • SUCCEED and FAIL
    • WAIT: wait for a certain period or time or until a date
    • CHOICE: allows a state machine to take a different path depending on an input
    • PARALLEL: allows to create parallel branches in a state machine
    • MAP: expects a list of things, for which it might perform a certain set of things
    • TASK: represents a single unit of work performed by a state machine. They can be integrated with: Lambda, Batch, DynamoDB, ECS, SNS, SQS, Glue, SageMaker, EMR, etc.

SWF - Simple Workflow Service

alt text

  • Allows us to build workflows used to coordinate activities over distributed components (example: order flow)
  • Allows to build complex, automated and human involved workflows
  • It is the predecessor of Step Functions, it uses instances/servers
  • It allows the usage of the same patterns/anti patterns like long running workflows
  • AWS recommends defaulting to Step Functions instead of SWF, use SWF only if there is very specific workflow that requires it
  • Workflow: set of activities that carry out some objective together with the logic that coordinates the activities
  • Within workflows we have activity task and activity workers
  • A worker is program that we create to perform tasks
  • Workflows have deciders, applications that run on an unit of compute of AWS
  • Deciders schedule activity tasks, provides input data to activity workers, processes events and ends the workflow when the object is completed
  • SWF workflows can run for max 1 year

SWF vs Step Functions

  • Default: Step Functions - they are serverless, they require lower admin overhead
  • AWS FLow Framework - way of defining workflows supported by SWF
  • External Signals to intervene in process, we need SWF
  • Launch child flows and have the processing return to parent, we need ot use SWF
  • Bespoke/complex decision logic: use SWF (custom decider application)
  • Mechanical Turk integration: use SWF (suggested AWS architecture)

Amazon Mechanical Turk

  • Provides a managed human task outsourcing system
  • Provides a set of APIs and a marketplace to outsource jobs to human beings
  • Allows requesters to post Human Intelligence Tasks (HITs) to a marketplace
  • Tasks are completed by workers, who earn money for it
  • Pay per task, perfect for tasks suited to humans rather than ML
  • Qualification: worker attribute, we can require a test. A qualification can be a requirement to complete HITs
  • Great for data collection, manual processing, image classification

Elastic Transcoder and AWS Elemental MediaConvert

  • MediaConvert a fairly new AWS product replacing Elastic Transcoder
  • MediaConvert is a superset of features provided by Transcoder
  • Both of the systems are file based video transcoding systems
  • They are serverless transcoding services for which we pay per use
  • For both we add jobs to pipelines (ET) or queues (MC)
  • Files are loaded from S3, processed and stored back to S3
  • MC supports EventBridge for job signalling
  • ET/MC architecture example: alt text
  • We use these products when we need to the media convert in a serverless, event-driven media processing pipeline
  • MC supports more codecs, design for larger volume and parallel processing
  • MC supports reserved pricing
  • ET required for WebM(VP8/VP9), animated GIF, MP3, Vorbis and WAV. For everything else we should use MediaConvert

AWS IOT

  • IOT - Internet of Things
  • AWS IOT is a product set in AWS, used for managing millions of IOT devices
  • IOT devices can be temp, wind, water sensors, light sensors, valve control sensors, etc.
  • All of these need to be registered into a system to allow secure communication for managing them: provisioning, updates and control
  • Communication to or from devices is likely to be unreliable, AWS provides device shadows: virtual representations of actual devices, having the same configuration registered for the actual device. We can read from them the last communicated data, essentially the device communicates with the shadow, the last registered data can be retrieved anytime afterwards
  • Device messages are sent JSON format, using MQTT protocols
  • AWS IOT provides rules: event-driven integration with other AWS Services
  • AWS IOT architecture: alt text

AWS Greengrass

  • AWS Greengrass is an extension of the services provided by AWS IOT, moving those services closer to the edge
  • Greengrass allow some services like compute, messages, data management, sync and ML capabilities to run from edge devices
  • Devices which Greengrass software can locally run Lambda functions, containers
  • Provides local device shadows which are synced back to AWS
  • Allows messaging using MQTT
  • Allows local hardware access for Lambda functions

AWS AppSync

  • AppSync is a managed service which uses GraphQL
  • It is used for building APIs on AWS which use GraphQL
  • GraphQL makes it easy for applications to get the exact data they need. This includes combining data from multiple resources
  • Datasets behind GraphQL can include:
    • NoSQL data stores
    • RDS databases
    • HTTP APIs
    • etc.
  • AppSync integrates with (resolvers) DynamoDB, Aurora, ElasticSearch, etc.
  • Supports customer resources using Lambda
  • Provides support for real time data retrieval using WebSocket or MQTT on WebSocket protocols
  • Mobile applications: replacement for Cognito Sync
  • Requires a GraphQL schema for getting started
  • Example for GraphQL schema:

    type Query {
        human(id: ID!): Human
    }
    
    type Human {
        name: String
        appearsIn: [Episode]
        starships: [Starship]
    }
    
    enum Episode {
        NEWHOPE
        EMPIRE
        JEDI
    }
    
    type Starship {
        name: String
    }
    

Security

  • Four ways we can authorize applications to interact with AppSync:
    • API_KEY
    • AWS_IAM
    • OPENID_CONNECT (OpenID Connect provider/ JWT)
    • AMAZON_COGNITO_USER_POOLS
  • For custom domain & HTTPS, use CloudFront in front of AppSync

AWS Greengrass

alt text

Ref

Bình luận


White
{{ comment.user.name }}
Bỏ hay Hay
{{comment.like_count}}
Male avatar
{{ comment_error }}
Hủy
   

Hiển thị thử

Chỉnh sửa

White

Nguyễn Huy Hoàng

17 bài viết.
10 người follow
Kipalog
{{userFollowed ? 'Following' : 'Follow'}}
Cùng một tác giả
White
11 4
(Ảnh) Tại hội nghị Build 2016 diễn ra từ ngày 30/3 đến hết ngày 1/4 ở San Francisco, Microsoft đã đưa ra 7 thông báo lớn, quan trọng và mang tầm c...
Nguyễn Huy Hoàng viết hơn 4 năm trước
11 4
White
7 0
Viết code chạy một cách trơn tru ngay lần đầu tiên là một việc rất khó, thậm chí là bất khả thi. Do đó debug là một kỹ năng vô cùng quan trọng đối ...
Nguyễn Huy Hoàng viết hơn 4 năm trước
7 0
White
1 0
MultiFactor Authentication (MFA) Factor: different piece of evidence which proves the identity Factors: Knowledge: something we as users know: ...
Nguyễn Huy Hoàng viết 4 tháng trước
1 0
Bài viết liên quan
White
0 0
FSx FSx For Windows File Servers FSx for Windows are fully managed native Windows file servers/file shares Designed for integration with Wind...
Nguyễn Huy Hoàng viết 4 tháng trước
0 0
{{like_count}}

kipalog

{{ comment_count }}

bình luận

{{liked ? "Đã kipalog" : "Kipalog"}}


White
{{userFollowed ? 'Following' : 'Follow'}}
17 bài viết.
10 người follow

 Đầu mục bài viết

Vẫn còn nữa! x

Kipalog vẫn còn rất nhiều bài viết hay và chủ đề thú vị chờ bạn khám phá!