Brksec 2037

Download as pdf or txt
Download as pdf or txt
You are on page 1of 147

#CiscoLiveAPJC

Starlink Security Advanced

Andrew Benhase – Federal Architect


@CyberSecOps
BRKSEC-2037

#CiscoLiveAPJC
#CiscoLiveAPJC
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Cisco Webex App

Questions?
Use Cisco Webex App to chat
with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App

2 Click “Join the Discussion”

3 Install the Webex App or go directly to the Webex space Enter your personal notes here

4 Enter messages/questions in the Webex space

Webex spaces will be moderated


by the speaker until December 22, 2023. https://ciscolive.ciscoevents.com/ciscolivebot/#BRKSEC-2037

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
BRKSEC-2037 CL Room

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Cisco Starlink – External Teams Space
webexteams://im?space=740fe050-925f-11ee-bb7e-295d996ede61

Ready and Waiting for you


to join!

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Agenda

• Catch up on current events


• What is Starlink
• Starlink Security
• Deployment Models
• SDWAN
• Conclusion

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
What I do here @cisco
• Federal Security Architect
• At Cisco >24 years, supporting US Federal
Government
• 32 years primarily supporting US Defense, Civilian
and Intelligence Communities
• Deep focus on defensive cyber operations, @CyberSecOps
advanced encryption, making security work! @ThreatCowboy
• My first Networkers was in 1995… [email protected]

• https://www.linkedin.com/in/andrewbenhase/

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
From MLB to MEL

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Latest News Updates
(since last year)
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Credit: SpaceX #CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Services Changes – Public IP

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
#CiscoLiveAPJC © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Cats love Starlink

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
How do you keep up?
Starlink Federal Room (SL-OSINT)

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Whats happening with
Kuiper?
First Kuiper Launch

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
December 2 2023

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Whats happening with
OneWeb?
OneWeb- catering towards Enterprise
• Polar Orbits
• Higher Orbit
• >600 satellites in orbit
• Broad Coverage over Australia
• Focused towards Enterprise Networking

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
What really is Starlink?
Starlink Australia
• Estimation of pathing
using currently published
ephemeris
• Broad coverage of urban
areas
• Rural areas would be
serviced by polar orbits
• Coverage in remote area
may be inconsistent

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Satellite v1.5
• Each satellite features four antennas in Ku band, one for uplink,
three for downlink
• Each antenna is capable of projecting eight beams in two
polarizations (RHCP/LHCP), for a total 48 downlink beams and 16
uplink beams.
• The maximum bandwidth available to Starlink in Ku band is 8x 250
MHz channels in downlink (total 2 GHz), and 8x 62.5 MHz channels
in uplink (total 500 MHz)
• Each Satellite nominally operates at 10Gbps capacity with future
expansion to 20Gbps

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Faced towards Ground

2 1 Downlink Antenna Cluster

3 4 Uplink Antenna

Credit: Starlink

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Satellite v2.0 and v2.0 Mini
• Requires Starship to realistically launch volume
• Starshield requires it
• Much larger payload (1800 lbs)
• Gen 2 Mini launch, critical failures, 50% de-orbited on initial launch
• Gen 2.0 satellites will take many years to fully field

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
V2.0 Satellite

V1.5 Satellite

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Observations of the Starlink Network
• CGNAT Employed • Exit Path is currently static
based on your Service Class
• Array to Satellite to Ground
Station are all Flat • Portabililty, Marine, RV, Aviation
means that you can be placed
• Appears that Ground to NAP is
in different exit VPNs, we
a series of Exit MPLS Networks
assume dynamically
• Exit Routing is based on your
specific Terminal
• *Network Configuration
changes are frequent and
unannounced
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Infrared “Space Lasers”
• 3 Beam Optical Head using
Infrared Laser
• Same Orbital Plane Operation
• Theoretically could offload to
parallel polar plane satellite

Credit: SpaceX/Starlink

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Ground Station
• Each gateway antenna has available a maximum of 4x
500 MHz channels (total 2 GHz) in uplink, and 5x 250
MHz channels (total 1.25 GHz) in downlink
• In this configuration – where 8 antennas are active –
would be 10Ghz total active Down and 6Ghz Up per site
• Ground stations are positioned on top of existing Fiber
Paths
• Each Parabolic Antenna can support 10Gbps x 2)

• So that’s up to 1.6Tbps theoretical bandwidth for a site


with 8 active

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Basic Networking

Internet Dynamic IP
address
assigned by
Starlink

NAT Router
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
List of Australian Starlink Ground Stations
Cataby, WA
Merredin, WA
South West of Coolgardia, WA
Wagin, WA
Ki Ki, SA
Pimba, SA
Broken Hill, NSW
Boorowa, NSW
Calrossie, NSW
Canyonleigh, NSW
Cobargo, NSW
Springbrook Creek, NSW
Tea Gardens, NSW
Ki Ki, SA
Anankie, VIC
Koonwarra, VIC
Torrumbarry, VIC
West of Emerald, QLD
Toonpan, QLD
Warra, QLD
Willows, QLD
Bulla Bulling, WA

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Lithuania

Radio Gateways

Lithuania
Poland
Poland
Internet Gateways Amsterdam
London
Frankfurt
London
How does it work in Ukraine?
Frankfurt

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
Double/Triple
NAT
Double-NAT – who cares?
• Your VPN cares, depending on
what you’re using….
Internal IP Address

Publicly Routed IP Address

NAT-1
NAT-2

CGNAT Router

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Things that Fix NAT Problems
• Static NAT configuration – impossible with Starlink and CG-NAT
carriers
• GRE/IPSec+NAT-T Tunnels
• Straight NAT-T Tunnels
• IPv6**** (maybe)
• TCP VPN Tunnels

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
Polar Orbits and
“Space Lasers”
Satellite Truths and Myths
• All Starlink Satellites have “Lasers” – FALSE

• All Starlink Satellites can cross communicate to each other – FALSE

• Some Starlink Satellites have laser based optics that can point
ahead of them to the next satellite – TRUE

• On-orbit Satellites can calculate multi-planar ephemeris to


dynamically communicate to satellites in different orbits - FALSE

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Polar Orbit Satellites and Free Space Optics
• Generation 1 Satellites are Radio Only
• Generation 1.5 and 2.0 Satellites are capable of Inter-Satellite Links (ISL)
• ISL Links work currently in a follow-me configuration
• A polar string of satellites provide hop to hop communications in single file
• Closest Radio Gateway provides the downlink for the chain of satellites
• Only use for satellites in polar orbits and where there is a Gateway
connection
• You may not pop-out onto the Internet in a country that you expect

• You may not come out in a country you want


#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Space Lasers fix my
networking issues
O rbit
ar
Pol

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Polar
Satellite
Train with
ISL

Gateway
Connection to
Montes Claros
Gateway

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
ay
w
te
Ga
os
ar
Cl
te
on
M
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
i te
f Satell
to
o o tprin Ground gre
io F io Ale
Rad ith Rad Porto
w to
ec ti on
n
C on

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
This satellite provides
the backhaul to each
of the other satellites –
selection algorithm is
unknown

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Challenges with
Polar Orbits
No Orbital Paths
No Radio Footprints here

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
Starlink Security
Sum Total of available Security

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
Starlink Security

BYOS – Bring Your Own Security


There is “very limited" security other
than what you bring yourself

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Starlink Router
• Nmap scan report for 192.168.1.1

• Host is up (0.0040s latency).

• Not shown: 994 filtered tcp ports (no-response)

• PORT STATE SERVICE

• 22/tcp open ssh

• 53/tcp open domain

• 80/tcp open http

• 9000/tcp open cslistener

• 9001/tcp open tor-orport

• 9002/tcp open dynamid

• Nmap done: 1 IP address (1 host up) scanned in 45.68


seconds

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
Things we know about Starlink Network
• Carrier Grade NAT (CGNAT) at the Internet Gateway
• IPv4 DHCP is assigned across the network
• IPv6 Prefix Delegation works on some Gateways
• Layer 2 network from terminal to ground to exit point (MPLS)
• Native IPSec will not work (CGNAT)
• IPSec Encapsulation works – NAT-T (udp4500)
• TLS VPNs work
• There is NO local NAT configuration possible on the SL Router

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Starlink Security Today
WPA2 Implemented here

CGNAT Router

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
Starlink Security Today
Security Must be
Implemented Here

CGNAT Router

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
Advanced Networking
Layer 2 Network

Radio Footprint of the Satellite

Router Placed
SL Router in Bypass Mode: Outside DHCP in Bypass Mode
gets assigned
WIFI Gets disabled by Starlink
Router is no longer locally accessible
Statistics are stored in SL Cloud
Array connects to SL Cloud and delivers updates

Firewall SL Ethernet
Adapter

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
Cisco Security + Starlink
Meraki+Firepower Deployment
• MX Series
• MR Series
• Z3 Series
• Firepower 1010 Series

*Transparent Inline Pair

*Planned for FDM in 7.4 Release


#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
Meraki MX/MR/Z3 Deployment
• MX Series
• MR Series
• Z3 Series

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
Native IPv6 Support on MX and MR Platforms

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
Meraki is the simplest security option

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
Meraki is the simplest IPv6 Deployment Option
• Takes the downstream
Prefix Delegation
• Automatically deploys it to
the downstream networks
• Clients will be assigned /96 IPv6 Assigned Interface
IPv6 address out of your
assigned Prefix

/64 Deployed Network

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
Actual Working Things
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
Makes forwarding decision based on API feedback

CG-NAT IP

Actually does API lookups


#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
Deployment
Considerations
QoS Observations
• Terminals (arrays) are statically
linked to what we believe to be
Scavenger class QoS
MPLS VPNs with static exits to
the Internet

Internet NAP
Scavenger Class
Premium Class
CGNAT Router Residential QoS

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
Observed Current QoS
Mobile (Best Effort) $$
Priority Data (EF) $$$
Residential (AF) $

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
Deployment Considerations
• You will probably want a 150 foot cable
• You can make a 300+ foot cable easily by inserting Ethernet in
the middle
• Use High Quality watertight connectors

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 108
Decisions
• “High Performance” is simply
double the array
• They have a single GigE output
– but have doubled the
transceivers
• They are clearly creating a
Service Class for High
Performance users and doing
traffic engineering to support it

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 109
High Performance
Terminal Considerations
HP Wires are 22AWG!

HP pin / Pot / Dish Wire Color / RJ-45 pin &


Color using 568B standard
A1 + Green 1 Orange/White
A2 + Yellow 2 Orange
A3 + Blue 3 Green/White
A4 + White 6 Green
B1 - Orange 7 Brown/White
B2 - Purple 8 Brown
В3 - Brown 5 Blue/White
B4 - Gray 4 Blue
B5 × Shield / Shell

CAT8 22AWG STP RJ45

*Courtesy: Lyle Tanner


#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
Austere
Deployment
Options
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 114
Disclaimer

I make 100% zero guarantees or warranty


you won’t damage something….

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 115
Ditching the Starlink Router

• https://dishypowa.com
• 48-56V DC passive PoE
injector
• Allows you to remove
Starlink Router entirely
• Connect up BYOS
options
• Needs 48v DC Power
Credit: dishypowa.com

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 116
Parts Needed

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 117
Wiring Diagram 100-200w Panel

MPPE Charge
Controller

12volt DC
LiFePO4 Battery
_ +

_
Inline 12volt-48v DC
Fuse +DC Boost Converter
Carefully evaluate your actual WIFI Power Requirements!

Minimum 120watts
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
Cisco Crisis Response
Deployment to Maui

-Requested to support Maui


First Responders with
Internet access
-Cisco Employees
dispatched with High
Performance Terminals
-Setup instant Internet
access

#CiscoLiveAPJC © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
SOLAR POWER
(last year)
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 122
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 123
SOLAR POWER
(this year)
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 125
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 126
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 127
Power/Solar Conclusions (from last year)

1 2 3 4
You will need You will need You will have to You will have to
more stored more solar trial 24hr account for lack
power than you power than you operation to be of full solar
think think sure it works cycles

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 128
Updated Solar Guidance
• Minimum 1000Wh between solar cycles
• Gen 2 dish consumes between 430-50watts consistently
• 200watts of solar panel will reliably recharge 800Wh in 4-5 hours
of direct sun
• 24 hour remote operation is completely achievable

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 129
Debugging
Debugging at the CLI

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 131
Docker Tools Repository

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 132
Remote Connections
• Performance Data is stored
in the Starlink Cloud
• Allows remote access to
data statistics from your
local network without being
there

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 133
http://dishy.starlink.com

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 134
• • "alerts": {
"hardwareVersion": "rev3_proto2",
"auth": {

Starlink Debug
• "motorsStuck": false,

• "softwareVersion": "223c055e-8fe8-42e6-8d00-
"accessToken": "<len=848>",

cd4c6a466252.uterm.release",
• "thermalThrottle": false,

• • "refreshToken":
"manufacturedVersion": "", "<len=66>",
• "thermalShutdown": false,

• "dish": {
• • "accessTokenExpirationDate":
"generationNumber": 1665611831,

"2022-10-
"mastNotNearVertical": false,

• • "reachable": true, • 27T18:39:21Z",


• "countryCode": "US",
• "unexpectedLocation": false,

• • "service": "dish",
• • "idToken":
"utcOffsetS": -17999, "<len=723>",
• "slowEthernetSpeeds": false,

• • "cloud": false,
• • "tokenType":false,
"softwarePartitionsEqual": "Bearer" • "roaming": false,

• • "isDev": false,
• • "features": { • • "installPending": false,
• • "bootcount": 129,
• • "stowRequested": true, • • "isHeating": false,
• • "antiRollbackVersion": 0,
• • "unstow": true • • "powerSupplyThermalThrottle": false
• • "isHitl": false
• • }, • • },
• • },
• • "timestamp": 1666895243, • • "gpsStats": {
• "deviceState": {
"gpsStats": {

• • "deviceInfo": { • • "gpsValid": true,
• "uptimeS": 122693
"gpsValid": true,

• • "id": "ut01000000-00000000-0008d16e", • • "gpsSats": 16,
• },
"gpsSats": 16,

• • • • "noSatsAfterTtff": false

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 135
Should you leave your SL Router WIFI Enabled?
• The short answer is yes – primarily for local debug reasons
• Just don’t use it for actual production users
• It is not secured
• It is not configurable
• It is not a Firewall
• It is a very poor performing Access Point

BYOS – Bring Your Own Security


#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 136
What do you have
running?
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 138
What is running now?
• Meraki MX95 (IPv4 and IPv6)
• Meraki MX65 (IPv4 and IPv6)
• Peplink MAX Transit Duo Pro (IPv4)

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
Meraki Starlink

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 140
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 141
WAN Health using 1keye

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 142
Agent based Reporting for Thousand Eyes running Starlink across the Globe

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 143
Agent to ICMP IPv6 not working

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 144
IPv6 PD Renew Every 5 Minutes

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 145
SDWAN
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 147
Things we know
• CG-NAT is a reality for all Residential and Mobile Plans
• High Performance Array is allocated a public IP address
• This is NOT a static IP address!
• DHCP 5 minute renewal

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 148
Catalyst SD-WAN Controllers Starlink Reference Architecture • SD-WAN version 17.x
On Premise in DC or Cloud • Hub and spoke topology with
Hosted
Manage Validato
Internet ODT
Data Center 1 SaaS/IaaS
r r Data Center 2 • Per-tunnel/adaptive QoS
• Application Aware Routing
• IPSec tunnels to SIG
• DIA with FW/DNS-Sec
• WAN Opt/DRE (TBD)
Controller
s • FEC/Packet Dup (TBD)

Internet Private

Downlink Broadband
10.7GHz –
12.7GHz Uplink LTE
25-250Mbps 27.5GHz – 30
GHz
Uplink
14.0GHz – Downlink
14.5GHz 17.8 GHz – 19.3
2-20Mbps GHz
Single Router Site
Starlink Dual Router Site
Single Starlink Gateway Dual Starlink
Earth Station Hybrid Transport Site
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 149
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 150
SD WAN Configuration
interface Tunnel100201
interface GigabitEthernet0/0/0 no shutdown
ip unnumbered GigabitEthernet0/0/0
description Ethernet to Starlink LEO satellite
no ip clear-dont-fragment
no shutdown ip mtu 1400
tunnel source GigabitEthernet0/0/0
arp timeout 1200 tunnel destination dynamic
tunnel mode ipsec ipv4
ip address dhcp client-id GigabitEthernet0/0/0
tunnel protection ipsec profile if-ipsec201-ipsec-profile
no ip redirects tunnel vrf multiplexing
tunnel route-via GigabitEthernet0/0/0 mandatory
ip tcp adjust-mss 1360 exit
ip dhcp client default-router distance 1 interface Tunnel100202
no shutdown
ip mtu 1500 ip unnumbered GigabitEthernet0/0/0
no ip clear-dont-fragment
ip nat outside
ip mtu 1400
load-interval 30 tunnel source GigabitEthernet0/0/0
tunnel destination dynamic
mtu 1500
tunnel mode ipsec ipv4
negotiation auto tunnel protection ipsec profile if-ipsec202-ipsec-profile
tunnel vrf multiplexing
tunnel route-via GigabitEthernet0/0/0 mandatory
exit

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 151
Crypto Configuration
crypto ipsec transform-set if-ipsec201-ikev2-transform esp-null esp-sha-
crypto ikev2 policy policy1-global hmac
proposal p1-global mode tunnel
! !
crypto ikev2 profile if-ipsec201-ikev2-profile crypto ipsec transform-set if-ipsec202-ikev2-transform esp-null esp-sha-
no config-exchange request hmac
dpd 60 10 on-demand mode tunnel
dynamic !
lifetime 14400 crypto ipsec profile if-ipsec201-ipsec-profile
! set ikev2-profile if-ipsec201-ikev2-profile
crypto ikev2 profile if-ipsec202-ikev2-profile set transform-set if-ipsec201-ikev2-transform
no config-exchange request set security-association lifetime kilobytes disable
dpd 60 10 on-demand set security-association lifetime seconds 3600
dynamic set security-association replay window-size 1024
lifetime 14400 !
! crypto ipsec profile if-ipsec202-ipsec-profile
crypto ikev2 proposal p1-global set ikev2-profile if-ipsec202-ikev2-profile
encryption aes-cbc-128 aes-cbc-256 set transform-set if-ipsec202-ikev2-transform
group 14 15 16 19 20 21 set security-association lifetime kilobytes disable
set security-association lifetime seconds 3600
integrity sha1 sha256 sha384 sha512
set security-association replay window-size 1024

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 152
Design optimizations for
LEO Satellite Transport
Reducing SD-WAN control plane traffic on Satellite links
Problems
LEO satellite have low transmit speeds (2-20 Mbps up) relative to
download speeds of 25-220 Mbps down). SD-WAN control traffic can
consume a high proportion of this bandwidth in heavily meshed
topologies with default timers. This includes:
• BFD probes over each IPSec tunnel (2.2 Kbps per SD-WAN
tunnel)
• OMP hellos and updates to/from Catalyst controllers (up to 80
Kbps)
• Statistics upload to the Catalyst Manager (up to 1.2 Mbps)

Control traffic is automatically mapped to Q0 on the WAN edge, which


can contend with user realtime traffic also in Q0 resulting in drops and
instability

Solutions
• Dynamic OnDemand tunnel design (reduces # of BFD sessions)
• BFD low bandwidth link
• Last-resort-circuit in cases where Starlink is for backup
• vManage connection preference 1
• Administration Settings for Statistics – disable some, all or vAnalytics only?
• QoS design with Adaptive QoS and 2-level policer / Split LLQ

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 154
Tunnel Optimizations for low bandwidth links

Low bandwidth link: Reduces BFD sdwan


overhead by 50% per tunnel interface GigabitEthernet 0/1/0
Last-Resort-Circuit: No tunnel or traffic unless all Description connected to Starlink
tunnel-interface
other transports down encapsulation ipsec weight 1
no border
vManage connection preference 1: Prefer color
terrestrial (if available) over Satellite for vManage no last-resort-circuit
low-bandwidth-link
control connections to reduce overhead of hello-interval 6000
hello-tolerance 600
statistics publishing. vmanage-connection-preference 1
OMP Hello tuning: Reduce Hello-interval
to further optimize bandwidth

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 155
Validation
Cat Manager
152.22.241.1
Validator
152.22.241.1
Cisco lab, Building 11-183, Lab
RTP, NC C8300-1N1S-
56 55
4T2X
DC1 FLM272112R8
SiteID C1121X-8P
FGL2624L51Q

SpaceX office, Redmond


Controllers
152.22.241.1 WA
55 Site ID 303301
MCNC
NCREN
Internet

SpaceX
Satellite
AT&T Backhaul
Cisco SP Lab, SJC Bldg
broadband AT&T 16
Internet 4G/LT
E

1121X-8P 1121X-8P
C1111
C8300 INIS
Cisco IoT lab, RTP Building 6
Tom’s house,4t2X
Wilmington, NC
Site ID 206202 #CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 156
Ground/Space Architectures
Recommended Edge Architecture

No Starlink Router NAT

Admin Only
Transparent Inline Pair

Each device gets its own IP from SL

Not Bypassed
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 158
Recommended Edge Architecture

FPR1010
Admin Only

Transparent Inline Pair

Catalyst 9200 Compact

MX Series

Not Bypassed
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 159
Recommended Core Architecture

Internet

Not a single class anymore

CG NAT
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 160
Recommended Core Architecture
Mobile (Best Effort) $$
Priority Data (EF) $$$
Residential (AF) $

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 161
Recommended Architecture

*Transparent Inline Pair

FDM Support in 7.4.1


#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 162
Internet

AT&T Verizon
LTE LTE AT&T

T-Mobile

MODEM 1 MODEM 2

LTE Router

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 163
Highest Probability of
Netowrk Uptime
Internet

AT&T Verizon
LTE LTE AT&T

T-Mobile

MODEM 1 MODEM 2

LTE Router

HIGH PROBABILITY OF UPTIME

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 165
Internet Smoothing
CGNAT Router

CGNAT Router
CGNAT Router

AT&T Verizon
LTE LTE AT&T

T-Mobile

MODEM 1 MODEM 2

LTE Router

HIGHER PROBABILITY OF UPTIME

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 166
LTE Cat 7/14 or 5G Cat 20 Modem?
• Great question – do you think you’ll be close to a 5G network on a
consistent basis?

• If not, I would opt for LTE based cellular modems x 2

• With WAN Smoothing technologies, over multiple interfaces, outage


impacts are largely minimized
315Mbp

2Gbps
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 167
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 168
Please Fill Out The Survey!

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 169
Session Surveys
We would love to know your feedback on this session!
• Complete a minimum of four session surveys and the overall event surveys to claim
a Cisco Live T-Shirt

#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 171
• Visit the Cisco Showcase for
related demos

• Book your one-on-one


Meet the Expert meeting

• Attend the interactive education


with DevNet, Capture the Flag,
Continue and Walk-in Labs

your education • Visit the On-Demand Library


for more sessions at
www.CiscoLive.com/on-demand

BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 172
Thank you

#CiscoLiveAPJC
#CiscoLiveAPJC
#CiscoLiveAPJC
Local Launch Pictures from
Melbourne (Florida)
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 176
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 177
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 178
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 179
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 180
#CiscoLiveAPJC BRKSEC-2037 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 181

You might also like