Cloud Concepts

Payment options:

CapEx (Capital Expenditure) – paying in advance for resources (like server, storage). Similar to on-premise environment, prepurchasing. It’s constant value each period. Not flexible.
OpEx (Operational Expenditure)- paying for consumed resources as we use them. Payment dependent on consumption, not constant.

Categories of cloud services:

Shared responsibility model. Some responsibility on your side and some on cloud side. There are many layers:

Storage, Network, Server, VM, (in IaaS Azure is responsible for these layers, but the rest is on you)
OS, Runtime (.nET, Java including Serverless), ( in PaaS Azure is responsible for these layers as well),
App, Data.

On-premise. You are responsible for all there layers.
IaaS. Layers from Storage to VM are in cloud. Rest is yours (i.e. picking OS: win/linux?).
PaaS. Layers from Storage to Runtime are in cloud. On your side the App and Data. Serverless is also kind of PaaS (event happens [store into db, publish event into the queue] which triggers action in the cloud).
SaaS. All layers in the cloud. So the software is there, like Microsoft 365. You have no responsibility.

Types of Cloud Computing

public (i.e. Azure)
private
hybrid (use public and private)

Reliability and predictability

Reliability (niezawodność) – autohealing after failure; autoscale when traffic peak;
Predictability (przewidywalność) -;

Describe Azure Architecture and Services

Regions and Region Pairs

Region definition: group of data centers where data travels in <2ms latency between them. Choosing region considerations:

performance (pick location close to your customers)
law regulations
resiliency (use different regions to avoid regional outages)

Region pairs are a group of regions in the same political boundaries where data is replicated. It’s for disaster recovery – pairs must be far away for protection reasons, but close enough to be in the same law regulations. See cross-region replication table: https://docs.microsoft.com/en-us/azure/availability-zones/cross-region-replication-azure

There are different environments. There is the default Az Cloud (commercial), but also AzUsGov, AzChina, AzGermany (it’s about compliance).

Availability zones

Data centers need power, cooling, networking. Every (almost) region has 3 availability zones. Zone is separate building provided with resources (power, cooling, …). It’s for resiliency concerns to be protected from single building (data center) issues.

Some services are ‘zone redundant’ – redistributes resources through all zones. ‘Zonal’ services are just in single zone (1 or 2 or 3) for example single VM.

Notice there are 3 availability zones in region to pick, but it does not mean there are 3 buildings in region, (in reality there is much more).

Resource group

In a single resource group you can have multiple regions and multiple resource types (like VM, network interface, disk, public ip). Resource groups cannot be nested. You can move resources between groups. Resource group is not a boundary for services. Groups are for organizational purposes. In a group you should have resources which share lifecycle (like vm + disk). You can create policies on group (like use only specific region, create only specific services, budget limit).

Subscriptions

Agreement between user and MS with certain billing model.

one subscription trusts only one tenant in Azure Active Directory (Azure AD tenant = directory)
you can apply budget, roles, policies. These settings are inherited in resource group
there can be many resources groups in one subscription, resource group lives in subscription
there are plenty limits in subscription (like max 980 resource groups). For small company one subscription is enough. For example you can have Test and Prod subscriptions.

Management groups

It helps to manage group of subscriptions. Manage groups can be hierarchical (root -> dev / prod -> pl/ de/uk) To management group you can apply:

budget,
policy,
roles based access.

Resource manager

ARM – it’s an RESTful enpdoint, interaction layer. Whatever service you are using (via portal or cli) it goes through ARM.

Notice in portal there is cloud shell (web command-line, you don’t need to install anything to use it).

You can create specification in JSON file (ARM JSON template) and send it through ARM to create resource. Looking at resource via the portal you can export such JSON template which you could need to recreate such a resource.

BICEP file is more human friendly file which is translated to JSON ARM.

Azure Arc

Arc extends cloud control via ARM for other (not Azure) clouds or on-premise services. Means you can use ARM to manage on-premise, AWS, … cloud (?).

Resources required by VM

VM should live in resource group of existing subscription. For VM we need:

managed disk (to store OS)
data disks (optionally)
vNIC (Virtual Network Interface) to connect to VirtualNetwork (with up to 2 subnets)
Public IP (bind it to IP configuration)
Network Security Group (prefered at subnet level)

Core computing resources

IAAS like resources

VM Virtual machine – use it when you need full access to OS (to configure it in specific way). There are different SKUs (specification based on goal – memory optimised/ computing optimised/ …).

VMSS (VM Scale Set) – VM from template (which OS to use, VM config (how many CPU, RAM), scaling (min-max, how/when to add/ remove CPUs/RAM).

Azure Batch – submit the job, azure will configure VM for you.

PAAS like resources

ACI Azure Container Instances – containers, you need to pick container image (public or your own) from registry. You can create container group. VM virtualizes hardware, but container virtualizes software.

AKS Azure Kubernetes Service – containers orchestration. In the background it creates VMSS for data.

Azure App Services – when you have an app to host (like API, web app, mobile app), it’s also kind of container. You just pick runtime, os and that’s it, push your code. In the background there is still some VM and you pay for it.

Function App (serverless) – run code you have written when something happens (on schedule, REST API call, ). Stateless code.

Logic App (serverless) – there is some state comparing to FunctionalApp. Don’t really need to be a developer to use it, there is graphical designer.

AVD Azure Virtual Desktop – provides desktop experience and publishes particular applications.

Core networking resources

Virtual Network resource exists in single subscription and region (cannot span regions) but spans Availability Zones. There must be at least one set of IPv4 addresses. Optionally you can add range of IPv6 addresses.

VN can be splited into subnets – portion of defined IP space (192.168.1.0/24 what gives 256-5 IP addresses). Azure internally needs 5 addresses for technical purposes (first Ip, last Ip, DNS x2, gateway). Then you can assign resources (like VM) to specific subnet.

Public IP Resource – add it to expose resource publicly. Usually we attach load balancer to public IP, less often VM.

Many VNets

If you have many VNets in many different regions always use unique IP ranges that they not overlap across different VNets. Then you can Peer them to allow connectivity between VNets.

To connect on premise environment and VNet in Azure there are two main options:

create VPN over the internet and in AZ you have to add VPN gateway and on premise you need to have VPN server. There are two types of VPN gateways: ‘policy based’ – for legacy solutions and ‘route based’ – default modern solution.
there are MeetMe points with ExpressRoute in AzureNetwork, it’s kind of private connection. Configuring ExpressRoute you decide what you want to connect with.

Resoucres outside of VNet – like Storage account.

Storage account can be i.e. a database. How to communicate to this from VNET:

In subnet you have to enable ServiceEndpoint for Storage. Then storage knows subnet and subnet knows storage. StorageAccount has an firewall which can allow connection from defined subnet.
Store account can be connected as a PrivateEndpoint. Subnet assigns single IP from it’s range to the storage (without using Public IP).

Public and Private Endpoint

public is internet routable. There is often firewall. You can make a ServiceEndpoint and make subnet a known entity to a public endpoint to allow it on firewall.
private – it’s an IP in the subnet. It’s a connection between resource and specific subnet

Security in VN

NSG – network security group, firewall-like rules, deny/allow specific IP/protocol.
Az Firewall – higher layer rules than NSG, filtering out specific hostnames, applications
DDoS protection – basic one is free, but works only for huge attacks, if you want something more customizable you need to go to standard plan.

Storage Account resources

It leaves in specific region. Would be nice to have storage account in the same region as computing item which uses it.

Performance options:

standard – all redundancy options (LRS, ZRS, GRS, GZRS),
premium – fast, but no redundancy except LRS.

Redundancy:

LRS – 3 copies of data in the same building (availability zone)
ZRS – 3 copies of data in 3 different buildings (availability zones)
GRS – 3 copies of data in the same AZ and next 3 copies in paired region (another AZ); not available in premium performance
GZRS – 3 copies in tree different AZs in the same region + other 3 in paired region; not available in premium performance.

Services types:

Blob – unstructured piece of data. Base structure for data lake.
- block – any type of unstructured data, lives in a container.
- page. There are disks with different types (i.e. premium ssd, standard hdd, ultradisk) and sizes; sizes can be changed during use.
- append (great for log)
Files – SMP/NFS protocols, sharing files
Queues – small messages, for event driven logic, FIFO
Tables – key-value pairs, schemaless

Block have access tiers (hot – constantly needed; cool – not often need; archived – not online, cheapest). Lifecycle of data – moving between tiers. In hot we pay less for transaction but more for space. On the other end in archive we pay less for space, but more for transactions.

Database Resources

Based on MS SQL Server offerings:

Azure SQL DB (PaaS service, multitenant).
Azure SQL MI (Managed Instance, works in your VNET, has more advanced features, use if you are moving from on-premise and use advanced features, there is webpage with comparison of features between Mi and Paas service).

Based on open source offerings:

MySQL
PostgreSQL
MariaDB

CosmosDB (no-sql, no-relational, multi models of data):

born in the cloud
multi-model (documents-mongodb, columns-cassandra, tables, graph-gremlin)
multi-consistency (how to copy data between instances: strong/ session/ eventual)

Data movement and migration options:

Online tools:

SMB file share on Windows Server -> in Azure it’s Azure Files. You can use Azure File Sync with SyncGroup. You can do cloud tiering (move not used recently files from on-premise server to Azure).
Azure Storage explorer – interactive UI in azure service
Azure Copy – to sync/copy data premise/azure and cloud/cloud, can be automated.
Azure Migrate – move VM with DB

Offline tools (for huge amount of data):

Azure Data Box:
- disk – they send you physical disk you copy the data and send back
- box – more or less the same
- box heavy – …

Azure Market Place

Products/offerings from Ms or 3rd party for easier deploy.

Products like VM machine with some specific software, like specific VPN, LAMP stack.

Sometimes charging is through Azure, sometimes somehow separately.

Azure IoT

IoT Hub – a service (kind of Paas solution) where devices can connect:

Telemetry – devices sending metrics to cloud
Devices uploading files
Cloud sends to device (firmware update, command)

DeviceTwin – representation of the device in cloud. Application in the cloud interacts with this twin using SDK rather than physical device.

You need to write the code which does your solution.

IoT central – kind of SaaS solution, uses IoT Hub. Software out of the box, when you don’t want to write your own app and play with SDK, just use ready dashboards, similarity to logical apps.

Dashboards/apps ready to use
Device templates – specific to some common industries
Simulates devices (for testing)
Handles common industry scenarios (customizable)
Rules to be defined (signal -> condition -> actions (sms/email/webhook sending). Example: If fridge temp is too high, send an sms.

Azure Sphere – focus on end to end security. It’s certificate based, not password based. There are 3 components:

AzSphere Microcontroler Unit (MCU).
Linux based OS
AS3 – AzSphere Security Service (making sure there is no malicious software, …)

Big data and Analytics Services

ETL:

extract (get raw data from any source. DataLake – storage type for unstructured raw data, cheap.),
transform (cleaning, deduplicating, wrangling – transform into new format),
load (into a sink – SqlDb, CosmosDb) to do analysis.

Az Data Factory – does ETL (from raw data to needed format), orchestrates all steps.

HDInsight – open source analytics services available to you (examples), transforming:

Hadoop – dividing tasks into smaller parts, disk based.
Storm – real time processing
Spark – batch jobs, data transformations, memory based.
Kafka – big data streaming
Hive LLAP – query data live from the store
HBase – noSql storage

Az Data Bricks – solution build on Apache Spark.

Az Synapse Analytics – workspace to manage ETL, business intelligence, etc., complete analytics solution (?).

AI Services

Training a model using existing data.

Az Machine Learning – platform for predictions basic on ‘historical’ data provided before. Allows total control for data scientists. Deployed algorithm is exposed via API and an app can communicate with it.

Az Cognitive Services – prebuild models, easy to spin up, not much coding, not much knowledge needed. Available services:

language
speech
vision

Az Bot Service – kind of virtual agent to communicate with user. Need to build knowledge base.

Serverless Technologies

Consumption based. Event driven.

Az Functions – runs piece of code in supported stack (.Net, node.js, Java, Python). It’s stateless. Durable functions can have some state.

Az Logic Apps – no/low code (you don’t have your code to run). There are some connectors between actions. There is designer. There are connectors for i.e. outlook, ftp, sql server, Az storage. There are some templates for specific scenarios.

DevOps Technologies

Az DevOps components:

Repository – place to store the code, GIT
Boards – tracking project state, i.e. KanBan board (JIRA alternative)
Pipelines – CI (integrating, testing code)/CD (deploying code)
Artifact – compiled image

GitHub – better alternative to Az DevOps, MS bought GitHub and do not invest more in AzDevOps:

Repositories – rich features here, code analysis, etc.
Board – JIRA alternative
Actions – CI/CD and much more
Projects – boards, weak today, no artifacts yet.

Today you should probably mix GitHub with AzDevops (repo in GH, board in AzDevOps).

Az DevTest Labs – environment, for testing builds in, create and after test destroy.

Az Management Solutions

Interactions with AZ: Azure <- Az Resource Manager (ARM) <- tools <- user

Tools:

web portal
mobile app
Az Powershell Module (i.e. get-azvm)
Az CLI [bash] (i.e. az vm list –output table)

Az Advisor

Free service with recommendations (cost, security, reliability, performance, …). Alerts can be created. There is general score, how good your solutions are.

Az Resource Manager (ARM) Templates

Declarative (describes final required state of the resource) json template. Template can be deployed to Az (i.e. via CLI). It’s idempotent, so if you deploy many times the same template, the final state will be the same.

There is also BICEP format, human friendly TypeScript format, also declarative. It is transcompiled to json templates behind the scenes.

There is also TerraForm template (declarative templates for different clouds, not only Az).

Az Monitor

There are metrics in time-based database. There are metrics per resource.

Logs you can configure in Diagnostics Settings. You can store logs in Az Storage (just to store) or send them to EventHub (to pass to external analytics tool) or to Logs Analytics Workspace.

Alert Rules – you can use logs and metrics as an input. You can define Action Rules to react (send sms, email, webhook, logic app).

Az Service Health

In the portal it’s hidden in ‘Help + support’ menu item. It displays recently found issues. Planned maintenance, Security advisories, Health advisories. You can create health alerts.

General security and network security

Ms Defender for Cloud (previously: Az Security Center)

There is overall security score/chart. There are recommendations.

You can add compliance policies (not only use the standard Az one).

There are defender plans (additionally paid services) to enhance security.

Key Vault

An app may need a secret (password, token, …).

Key Vault supports:

secret (sth you can read and write, i.e. password)
key (generate or import, cannot get it out of KeyVault)
certificate (lifecycle management, distribution)

There is role based access to KeyVault secrets. You can have App Service resource which can have right to read some secrets from key vault.

Az Sentinel

Sentinel is a service to detect security issue and automatically react on them. Sentinel sits on top of Log Analytics Workspace. There are two subsystems:

SIEM – Security Information and Event Management. Detecting and looking into the problem. You can use many external connectors to collect information. There is a list of incidents.
SOAR – automatically responds to an event.

Az Dedicated Hosts

It’s an concept for isolation, when you want to avoid VMs of other companies. You can create host group and put in there dedicated hosts. In host you can create virtual machines. Hosts have different types. Having dedicated host you can control a little maintenance window. On Dedicated Host you can place many VMs (big or small) but all of them must be of the same SKU.

Defense in Depth

There are security layers (even if one is compromised, the next still works). You need to care about specific layers depending on your solution (IAAS, PAAS, SAAS, cloud, on premise). There is Ms Defender to help with security.

Security layers:

data
application
compute (i.e. vm)
network (allow only specific IP)
perimeter (?)
identity + access
physical security

Security aspects (CIA):

confidentiality
integrity
availability

Zero Trust

Zero trust principles:

Verify explicitly (revalidate every request whether it’s allowed)
Least privilege (the least right as possible)
Assume breach (we assume incoming signals are corrupted)

Components:

identity (passwordless, single sign on, 2FA)
endpoint (users laptop, IoT device, equipment)
network (where request is coming from, encrypting communication
context (ctx of the request, understand overal risk of the request, conditional access)

Network Security Groups NSG

VNet exists in specific region (cannot span). VNet can be divided into subnets.

NSG must be in the same subscription as VNET. It has following properties: rules, names, priority, IP port, protocol, action (allow/deny).

Thanks to NSG you can define granular rules about network traffic (what to allow). By default all inbound traffic from the Internet is denied. Kind of firewall rules. NSG has to be linked to subnet where these rules are applied. So you can restrict traffic between subnets or peer network or from on premise network. NSG understands TCP/IP layer.

Az firewall

Az firewall is deployed in it’s private subnet (AzureFirewallSubnet). Traffic is going through this firewall. You can set application rules (urls) and network rules (ip addresses). Az firewall scales automatically to handle whole traffic.

Az DDoS Protection

DDoS types of attacks:

volumetric (so high traffic)
protocol (malformed packages, ping of death)
application (high traffic to slow down reads/writes, …)

Protections:

basic DDos Protection (huge scale attacks, you don’t have insights here). It’s free.
standard DDoS Plan – you can create it and link it to you VNets. It’s paid. There is rich reporting, metrics, adaptive tuning, rapid response, credit (scaling when attach can not be handled). You can use plan only for resources living in a VNet.

Azure Identity

Authentication and Authorization

Authentication (AuthN) – proving that user/service really is who they say they are. Ways (what we know, what we are, what we have). I.e. protocols: OIDC, SAML, WS-FED.

Authorization (AuthZ) – defines what user/service is allowed to do (roles). I.e. protocols: OAuth2.0

Each resource (i.e. VM) has Access Control and in it 3 generic roles (Owner – can do everything; Contributor – like owner, but cannot change privileges; Reader – cannot change anything).

Az Active Directory (AD)

IdP – Identity provider for the cloud. There is no group policy, hierarchy in the cloud.

Different apps/services just trust Az AD (office365, …). There can be single-sign-on experience.

AAD Connect – synchronizes roles from your local, on-premises AD to Azure AD.

Conditional Access, MFA, SSO

Conditional access – it’s through policies, conditions to be met by user/device. (Az AD -> Security -> Conditional access in portal menu). Then you can grant/block access.

MFA – multi factor authentication (two or more of these: we know/ we are/ we have). By default you have access to MFA service through AuthApp. If you buy premium plan, you can configure it an use other ways to MFA.

SSO – single sign on – I authenticated once to AzAD. When I access other apps, I don’t get prompted again. Other apps trust in AzAD.

Role Based Access Control RBAC

Role assignment – for certain identity I have certain role (group of actions/permissions) being assigned to certain scope (management group, subscription, resource group, resource). Role assignments are inherited, so if you set role on subscription, all resources in it will inherit it.

Generic roles:

owner – can do everything
contributor – can do everything, but cannot grant access
reader – can see, but not edit

Resource Locks

Resource lock can be applied on different levels (resource, resource group, subscription). It’s inherited as role assignments. To apply resource lock you need to be the owner of resource or administrator. To delete a resource, you need to remove lock first and only then you could delete it.

Lock types:

cannot delete – you can modify, but cannot delete
read only – you cannot modify, nor modify

Az Policy

Policy let’s define thing I need (rules + actions). Can be applied to different levels (management group, subs, …), it’s inherited. Initiative is a set of policies. Policy defines my guard rails. Policy – there is some condition, if it’s met, use an effect.

Role – means who can do something. Policy – means what can be done. Initiative – group of policies

Effects:

audit (do not deny if condition is not met, but calculate percentage of conditions are not met, for further information)
deny (deny action if not met conditions)
append/modify (i.e. add tag)
ifNotExist

For example you can define which sku’s (condition) can be used while creating storage resource and if somebody uses different one (condition not met), just deny creation (effect). You could assign such policy to subscription.

Governance hierarchy constructs

Management hierarchy. There is top/ root management group. Here starts hierarchy or management groups. Up to 6 levels of such hierarchy. Below these groups you can have subscriptions. In subscriptions there can be resource groups with resources. You cannot nest subscriptions in subscriptions (the same about resource groups).

Mngmnt group – is any group you create for organizational purposes, which help you organize all resources (i.e. prod, test, poland, europe, HrDepartment, …).

With these 3 constructs (mngmnt group, res group, subscription) you can do common things:

use roles based access control, assign to scope (who can do certain actions)
assign policy to scope (what actions can be done)
set budged – $ or %, actual or forecast (how much can it spend)
use inheritance (top most scope has loose settings and each group down in the hierarchy more and more strict)

Az Blueprints

While creating Resource Group we should have following artifacts:

arm template
RBAC
Policy

We can define a blueprint (kind of definition) from these items and assign this blueprint to subscription.

It can be defined whether during assignment something can be changed or it’s locked.

It can say that: assign some rights, add some tags, do not lock, etc.

Cloud Adoption Framework

Set of best practices when working with cloud. It’s a ms webpage, help with getting started.

Privacy and compliance resources

Important documents:

Ms Privacy Statement – how ms collect and uses data.

Online Services Term – key agreement between Ms and customer.

Data Protection Addendum – data security, data retention and deletion.

Trust Center – hub to get information about: security, privacy, compliance.

Sovereign Regions – there are a few totally separated Azure clouds (main, US gov, china, germany).

Cost management and SLA

Factors that affect cost

There are different types of resources and different costs. There are different SKUs, tiers, location also influences cost.

Meters:

exists (if load balancer/ public IP exists – you are paying)
running (if VM exists – you are paying)
instances (how many, autoscaling)
work (i.e. serverless (az functions, az logic apps. you pay when work is being done)
storage (db; on data used or data provisioned)
interactions (transactions against resource)
licences (windows licenses, oracle licenses)

Factors to reduce cost

How to save money:

Use autoscale to use as less instances as possible.
Use serverless to pay only for work done.
Pick correct SKU (cpu/memory)
Deallocate (VM have autoshutdown option)
Delete resources when not required
Choose correct tier, i.e. regarding storage pick one of hot/cool/archive
Tag VM with owner/ project to know who is responsible for specific resource, to know whether you can delete it.
Use Az Advisor – there are cost optimization ideas.
Az Reservation – is a commitment that you will use in next 1-3 years specific amount of resources. It gives you huge discount (30-60%), but even if you use less, you pay the specific money.
Use already bought licenses (i.e. Windows, SQL server) in the cloud, it’s called ‘Az Hybrid Benefit‘.
Az Spare Capacity – you can use this if your job can be stopped and resumed later. You can use cheap capacity then up to the moment when it’s price rises (because of the demand). While creating this you specify max price you want to pay. Cost can depend on region.

Pricing and TCO Calculators

There is pricing calculator where you can pick any services you plan to use (i.e. VM, disks, licenses) and it will calculate total cost. It just helps to estimate cost.

TCO – total cost of ownership. It estimates how much it costs to have on-premises environment and how much could you save when moving into cloud.

Az Cost Management

Cost analysis – shows how much money you have spent so far and a prognosis for selected subscription. Shows which types of services cost you the most. There is a nice flexible chart.

Budgets – let you define a number ($) and fire an action when this value (money) is spent or forecast reaches it. For example send an email if you reach 200$ per month.

Az Service Level Agreement

SLA – is the commitment from cloud provider regarding availability.

SLA levels:

99 – it’s 99% availability (it means 1,68h downtime possible per week)
99,9 – 10,1 minutes downtime (for VM to reach this level you need to have just one VM instance, but with premium SSD disk or Ultra Disk).
99,95 – 5 minutes (for VM to reach this level you need to have at least two VMs deployed in the same Availability Set)
99,99 – 1,01 minute downtime (for VM to reach this level you need to have at least two VMs deployed across at least two Availability Zones in the same region)
99,999 – 6 seconds downtime a week

There is Az Status board where you can take a look whether service was affected (http://status.azure.com/status).

Composite SLA – situation where you have two separate resources and for each of them you have SLA 99,9 so composite SLA will be lower, because both services can be down in different point of time.

If service is free, there is no SLA.

Az Service Lifecycle

Lifecycle stages (when new feature is added to Az):

Internal
Private Preview – limited set of customers
Public Preview – open
Generally Available (GA), have SLA support

There is Azure blog with info about new features you could use/ preview and report feedback.

—

_https://www.youtube.com/watch?v=pY0LnKiDwRA&list=PLlVtbbG169nED0_vMEniWBQjSoxTsBYS3 62

_https://www.youtube.com/watch?v=tQp1YkB2Tgs 2:45:00