AWS - Cross Account access using #IAM

Access multiple AWS accounts

Amazon says that you should have an account for different services, projects and clients.

This can get confusing very quickly, as you may end up with many many user ids and passwords.

There are many tutorials and follow-me videos on this topic but they can very confusing.

Personally I have found these two useful:

- https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html

- https://www.youtube.com/watch?v=n1r9Fp7GKvk

How to do it?

Let's assume you have two accounts ROOT and CLIENT. (Most examples have dev and prod and the like, but that does not give the relationship).

The ROOT account is where you control all the users and the account that is your main account. The account that pays for everything.

The CLIENT project could be where you host some lambdas for a specific client. An account you want to keep separate.

Step #1 - Root account number

Take note of your ROOT account number.
This can be found at the top of this page in the console: https://console.aws.amazon.com/billing/home?#/account

Do the same for your other account.

Step #2 - Set up cross-account access in the ROOT

Log into your ROOT account.

In IAMS create a policy called CrossAccountAssumeRole.

All this role needs to do is allow all actions on STS and for all resources.

Create a group called CrossAccount with this role attached to it.

Place the users who you wish to be allowed cross-account access the new group.

Step #3 - Allow your other account to connect to the ROOT

Log in to your other account CLIENT.

Create a new policy for your cross-account user to be assigned when it assumes its role.

This is important as this is the role that will define what your other account is allowed to do.

It will not be a Group or User ... its a role!

A useful AWS defined policy for cross-account admin is ... AdminsitratorAccess ... so we will use this.

Click create Role;

Select the trusted entity as "Another AWS account";

This will ask you for an account id. This is the id of the ROOT account (step #1).

This is important as it is to establish trust between the CLIENT and the ROOT. 

And no I don't know why sub-accounts don't trust the ROOT account implicitly.

There are two checkboxes on that screen for MFA and external ID ... either ignore them or add MFA.

Attach as many or as few policies as you need the role to have.

You will be asked what the users who can have this are ... ignore this for now.

Call the role "MainAccountAdminRole".

You can create many roles this way and as you might create User groups the Roles can be for any purposes.

Step #4 - Assume the Position role

Now log out and back into the ROOT account;

On the menu bar under your login name will be the option "Switch Role";

You will be presented with a screen asking for Account.
This is the account CLIENT Id from step #1, ... type it in.

You are asked for the role ... this is the role you just created in step #3, .. "MainAccountAdminRole".

Add a nice display name ... "Client" and a colour.

Then click switch role.

And you are in!

Beware!

There are some bear traps along the way here so be mindful:

- The role in the CLIENT account can do anything it is assigned to do. So admin access could be dangerous

- The "Switch role" settings exist in your browser cache. So you will need to set up the switch roles for new browsers, etc.

-- Enjoy!

Fixing your BASH colours

The default colours on Windows Linux Subsystem are terrible.
But the issue is a BASH problem and not a WLS issue.

To fix it to something nicer...

By default, the colours in windows 10 BASH shell are so terrible it's pretty hard to read the folder names. To fix this append the following two lines to your $HOME/.bashrc file.

LS_COLORS='rs=0:di=1;35:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=01;36;40:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:';
export LS_COLORS

If these changes are not to your liking check out these links:

• Fix Font Colors In Windows 10 Bash
• Bash for Windows colors are unreadable with default terminal theme

Gerkin, Cucumber & BDD

http://guide.agilealliance.org/guide/gwt.html

Given - When - Then

Definition

The Given-When-Then formula is a template intended to guide the writing of acceptance tests for a User Story:

• (Given) some context
• (When) some action is carried out
• (Then) a particular set of observable consequences should obtain

An example:

• Given my bank account is in credit, and I made no withdrawals recently,
• When I attempt to withdraw an amount less than my card's limit,
• Then the withdrawal should complete without errors or warnings

https://github.com/cucumber/cucumber-js
http://stackoverflow.com/questions/14638254/gherkin-to-not-for-javascript
http://blog.josephwilk.net/ruby/testing-javascript-with-cucumber-in-javascript.html
http://cucumber.github.io/cucumber-eclipse/
https://thomassundberg.wordpress.com/2014/05/29/cucumber-jvm-hello-world/
http://behat.readthedocs.org/en/v2.5/guides/1.gherkin.html
http://custardbelly.com/blog/blog-posts/2014/01/08/bdd-in-js-cucumberjs/
https://cukes.info/

The mystery of OAuth

The puzzle

Probably like many people I am perplexed by OAuth2.
No sooner do I think that I have worked out what it is I find that the next time I look it has changed or someone is describing it differently.

So this page is me pulling a few notes & sites together for reference.
If they help you then all to the best.

References

1. A good place to start is this talk on the topic:
   
   This guy manages to simplify the whole mystery.
   He also references two nice resources.
• OAuth 2.0 <debugger/> ... https://oauthdebugger.com/
• And a grant debugger (link todo)
2. A PHP libary but the documentation is surprisingly clear and easy to read.
   http://oauth2.thephpleague.com/
   The flow chart to select the grant type is especially good:
   
   The above can be found on http://oauth2.thephpleague.com/authorization-server/which-grant/
3. A second nice reference is the microsoft site: https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-v2-protocols
   This too has some useful & simple diagrams.
   Such as this one:
   
   Which shows a simplified relationship between servers.
4. 
   

Cloud Usage Guidance

Cloud environments are great - they enable us to do things faster, experiment, work collaboratively, etc, etc. However, without care, cloud environments can also become security risks and burn through cash quickly.

Two of the most important principals are:
Security: Apply a least-privilege approach to cloud resources, i.e. only allow access to the people that need it. This means consideration of ports, IP addresses and user permissions that need to be granted access
Cost: Understand the cost of resources that you're creating and ensure they are destroyed when they're no longer required and, ideally, turned off or scaled-down (vertically and horizontally) when not in use

The following sections provide some guidance to consider when creating and using cloud resources. The principals behind the recommendations are agnostic of the cloud provider, albeit the implementation might differ slightly for each and examples given are more focused on AWS.
The intention is for everyone to be aware of the various considerations that apply to the use of cloud environments, even if you're just spinning up a single VM to learn about the cloud.

Security

We are not re-inventing the wheel, as there are many existing security practices for AWS and Azure, e.g. for AWS, it's worth checking these out:
https://d0.awsstatic.com/whitepapers/aws-security-best-practices.pdf
http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html

Network Security Considerations

It is critical that network security controls, especially Security Groups (in AWS) and Network Security Groups (in Azure), are implemented correctly and consistently to avoid creating holes, or vulnerabilities, in the security posture of your environments. This will also ensure that in the nightmare scenario of a security breach of a VM, the damage can be as limited as possible.

A few considerations, which mostly relate to IaaS, but should be considered for all environments are:

• Traffic should only be permitted to travel within and between networks on a least-privilege basis, particularly for those environments that contain client data or are connected to a client  network, only granting access to those security groups, IPs and protocols that are known, understood and trusted
• There should never be any Security Groups created with the default 0.0.0.0/0 for ALL protocols for outbound traffic. This applies to public and private subnets. For example, if a VM was hacked, it would be harder (i.e. slow them down, if not stop them) for the attacker to jump from that VM to another or use it for other nefarious activities, if non-essential outbound ports are not permitted - considering that SG rules are stateful, it's unlikely that you'll need outbound SSH (port 22) permitted on any server other than a jump/bastion server.
• Routing tables should ensure that traffic is only routed between subnets and VPCs that require it. Where possible, Security Groups should reference other Security Groups, rather than IP addresses directly when setting rules for the same or peered VPCs
• Inbound security group rules should IP whitelist all traffic unless there is a very strong reason not to,
  AND a comment should be added to describe exactly what that IP refers to. Usually, all traffic originating from a Client's office network or the client VPN will have the same IP address, so this should be used on all externally accessible ports/endpoints.
• In a more mature environment, it is preferable for security group rules to be scripted and regularly reapplied/enforced via automation to force people to formalise and document network access in scripts any avoid manually added rules that could adversely affect the security posture of the environment.
• When designing environments, or even doing PoCs with any kind of sensitive data or other information, use subnets to segregate public and private networks and consider how the servers and services are accessed, e.g. via jump/bastion servers, rather than directly connecting all servers to the internet (in an ideal world, automation of deployment, configuration, log access, etc would eliminate virtually all need for a user to log on to a server)

IAM Best Practices

• For EC2 instances always use IAM roles, never use regular access keys.
• Always grant least privilege. Build your policies as specific as possible. Avoid wildcard permissions. Also, see http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege.
• Do not use IAM users when using an account that has company SSO setup to access AWS
• Follow or create naming conventions

VPC Best Practices

• Always assign security groups to instances
• Consider using existing security groups before creating new ones, however, only re-use SGs across similar instances/services to avoid confusion and inadvertently adversely affecting the security of another instance by changing a shared SG
• For Security Groups, open only ports you really need to be opened and restrict access to these ports
• Avoid exposing instances to the Internet whenever possible
• Consider enabling Flow Logs (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/flow-logs.html)
• Restrict egress traffic, avoid allowing outbound connections for any port to anywhere

EC2 Best Practices

• For opening access to the instance from the Internet use Elastic Load Balancer
• Always use HTTPS for publicly exposed HTTP-type services
• Enable access logs collection for your ELBs (https://aws.amazon.com/blogs/aws/access-logs-for-elastic-load-balancers/)
• For inter-service communication always route traffic only via a private network (meaning either instanceA→internal ELB→ instanceB or instanceA→InstanceB, but no instanceA→Public ELB→InstanceB)
• Use bastion servers to access instances via SSH/RDP

S3 Best Practices

• Always grant least privilege permissions to the bucket (http://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html)
• Enable access logging (http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html)
• Use encryption whenever possible and makes sense (http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingEncryption.html)
• For accessing S3 buckets from inside the private network, use VPC S3 endpoint (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-endpoints.html)

Cost

• Some cloud services are effectively free with low usages, such as AWS Lambdas and Azure Functions, but others aren't. Therefore, please be mindful of the cost of the resources being created. The considerations should include:
• The size of the resource (e.g. for a VM, it's the number of CPUs and size of RAM; for disks, it's the type/speed and capacity)
• Whether the resource is charged all the time (see below)
• Whether the resource can be scaled down / turned off, when not in use
• Whether there are more cost-effective ways of achieving the same thing

This does not mean don't try or use certain services, it just means be mindful of the consequences.

AWS and Azure provide comprehensive pricing pages and calculators to help with this.

For example, for VMs:

• https://azure.microsoft.com/en-gb/pricing/details/virtual-machines/
• https://aws.amazon.com/ec2/pricing/

As mentioned above, the pricing models for cloud resources vary, here are a few different types:

• Charged whilst running - VMs are a good example of this (but their storage is usually billable even when the VM is shut down)
• Charged when used - Lambdas / Functions are a good example
• Charged until destroyed - RDS is a good example

It is also worth checking whether there are any free credits that are available.

Thought of the day: Questions to ask your clients and developers?

There are times when it is a great idea to ask your clients and your developers a few questions about the project you are about to join.

Naturally, you will not be able to ask all the people the questions but you never know.

However, the answers should give you a rough idea of the maturity of the project.

The following table is an overview of the detail & questions you may need from each of the stakeholders. You may determine that additional/different people are needed to fulfil this and some people may have multiple roles but it should be a starting point.

Who (Client)	Information sought
Project Sponsor	Provide an overview of the Objectives and benefits expected What are the Business’ general expectations of the project? How are they being assessed? What are the Business’ technical expectations of the project? How are they being assessed? What are the Business’ security expectations of the project? How are they being assessed? Is there a feature project roadmap and is it shared with the Team?
Product Owner	Provide an overview of the prioritised delivery roadmap What are the Business’ general expectations of the project? What are the Business’ technical expectations of the project? What are the Business’ security expectations of the project? Is there a feature project roadmap and is it shared with the Team? What security requirements has the business placed on the system? What non-functional and performance requirements has the business placed on the system? What is the current and future growth of the system expected to be? Are requirements prioritised using a MoSCoW scheme? As PO are you involved in the Sprint planning phases of the project? Are you involved in the development of the project feature backlog? Are you involved in the development of the sprint planning process? Are technical blockers reported to you? Is there sufficient project information being provided by the team to assist with the creation of further requirements? Provide a walkthrough of the feature backlog and priority Provide insight into the current issue/risks/constraints to the project
Project Manager	Provide an overview of the current SDLC, measures and controls What are the processes & tools for requirements control? Does the project have a Release timeline? What is the process the team is following to control their development? If it is agile can you describe the process? Who is involved in the development of the project feature backlog and whats the process? Who is involved in the development of the sprint planning process and whats the process? As PM how would you describe your role in this SDLC? Does the project have a measurable sprint velocity? As PM are you involved in the Sprint planning phases of the project? How are testing outcomes & blockers/impediments reported and tracked? Are technical blockers reported to you?
Data/Business Analyst Business SME	What is the Business’ general expectations of the project? Provide an overview of the top priority Use Cases for the solution What are the processes the system needs to perform? What is the current quantity of data the system must process? What are the data sources, transformations and outputs the system must perform? (schemas, definitions, data flows etc) What security requirements has the business placed on the system? What performance requirements has the business placed on the system? What is the current and future growth of the system expected to be? How are the requirements communicated to the team? How is system functionality measured against the requirements?
Technical Architect	What is the overall System architecture context? What are the main functional components of the system? What are the processes & tools for requirements control? Are you involved in the development of the project feature backlog? Are you involved in the development of the sprint planning process? What security requirements were placed on the system? What performance requirements were placed on the system? What is the current and future growth of the system expected to be? What architecture requirements were placed on the system? What other non-functional requirements were placed on the system? Is there a Disasters recovery plan for the system? If so how is it implemented architecturally? Is it possible to get access to all existing design notes for the system? How are design choices and patterns communicated with the team for development? Is there a deployment strategy or process for the system? (Infrastructure as code?) Is there a testing/assurance strategy or process for the system? When there are issues with the development are you informed and consulted? When there are issues reported by assurance are you informed and consulted? Prior to the system going operational are you consulted by the Assurance, Operations or development team on how systems should be configured, or any potential issues? When in operation do you have visibility of operational monitoring systems? Once the system is deployed how do you know that it has been deployed according to your specification? Does the system have a ‘Technical Debt log’ or a list of known issues? Provide an overview of the solution architecture Provide details of constraints, dependencies and issues encountered and impacting delivery
Vendor Tech Lead	How are requirements communicated to the team? What is the process the team is following to control their development? If it is agile can you describe the process? Are you involved in the development of the sprint planning process? Is the team aware of the release schedule for the system? Is the team recording development information in some way? If so using which tool? What security requirements were placed on the system? What performance requirements were placed on the system? What architecture requirements were placed on the system? What is the process of reporting progress in development back to project management? What are the development toolsets in use across the team? When source code is developed what coding standards are being followed? When source code is being developed which source code management (SCM) tool is in use? Who has Source code access? When a build of the system is performed what are the build artefacts and how are they stored? What is your SCM versioning & branching & merging strategy? When defects are found, how are they reported, tracked and reported? What are the capabilities of development staff & their relative strengths? Is the team following a BDD/TDD development approach? What level of code review is performed during development? What is the release process for new functionality? Do you produce release notes for each new release? How many environments are used for development? What is the process for procuring cloud development stacks? What are the data sets used during development? How is the development environment secured? Is there a build process for the system? If so what is it? Is the system deployable using automation tools? If so what are they? What level of testing (manual & automated) exists for the system to prove functional & non-functional requirements? When the system is in UAT what information/documentation is requested by Assurance to determine the available functionality? When the system is in LIVE what information/documentation is requested by Operations regarding its operation and deployment? Provide an overview of the solution and the code Provide details of constraints, dependencies and issues encountered and impacting delivery
Security & Compliance	Provide an overview of the prioritised Security & Compliance requirements for the solution When the system requirements are being determined was security consulted? When the system is being developed what involvement does Security have? When the system deployment is being planned is Security consulted? Is Security informed of code related security issues? Is security informed of how the system is tested for compliance with security requirements? When operational how is the security of the system managed? If there is a security issue with the system how are you informed? What is your process for reporting a security breach? Provide an overview of constraints & dependencies for delivery
Assurance Lead	Provide an overview of the testing strategy and framework Provide an overview of the test coverage on the solution and the measures in place What involvement does assurance have in the code-Review process? What level of functional testing does assurance perform on the system? What level of non-functional testing does assurance perform on the system? What proportion of the testing is automated vs manual? Does the system have automated test reports and how are they reported to assurance What is the test plan for penetration testing? How does assurance get involved with the defect management process? Does assurance have any input on the architecture of the system? Once deployed what involvement does Assurance have with the system?
Operations	Provide an overview of the path to live/transition to live process. When the system is being designed what visibility do you have or need of its functional requirements? When the system is being designed what visibility do you have or need of any security requirements? When infrastructure is needed by the Development, Test and support team what is the process they must follow? How are you informed by development of the build artefacts needed for deployment? How do you enable or control the provisioning of cloud resources by the development team? What information do you need regarding the security of the proposed system? What information do you need about the testing process? How are deployment details of the Deployment & Configuration process communicated to you? What is the process for post-deployment system testing? How is performance and systems failure reported? What are the Monitoring processes and how & what is reported? If the system has a bug how is it reported and who does the repair work? What are the system’s supported hours? What is the systems supported availability? What is the DR plan for the system? When in operation does development have access to the system to perform fault analysis?

 Have fun!

A problem with AWS: IAM Policies

Today I had an issue where I wanted to create a Power Developer user.
So I promptly went to the managed roles and used the Developer Power User and clicked apply.
It seemed ok until I attempted to look at Lambdas and I hit issue after issue with missing roles.
So I have created this policy and it seems ok so far:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "NotAction": [
                "iam:*",
                "organizations:*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole",
                "iam:DeleteServiceLinkedRole",
                "iam:ListRoles",
                "iam:ListRolePolicies",
               "iam:GetRole",
               "iam:ListAttachedRolePolicies",
               "iam:GetPolicy",
               "iam:GetPolicyVersion",
                "organizations:DescribeOrganization"
            ],
            "Resource": "*"
        }
    ]
}

The items in bold are the additional rights the policy needed for the Lamda console to work.

Ideally, this should be 'LambdaConsoleRole' or some such but I am still learning.