CONFIDENTIAL C

Use
z.c al
tws enti
om
ng@ nfid
gyo Co
MT7986 HW QoS Application Note

din Tek
2022/01/20
ang edia
M

MediaTek Proprietary and

MediaTek Confidential.
Proprietary © 2021©MediaTek
and Confidential. Inc.
2021 MediaTek Inc.All
Allrights reserved.
rights reserved.
 CONFIDENTIAL C
Use
Version History

z.c al
tws enti
om
Version Date Author (Optional) Description
0.1 2021-04-20 Henry Yen Initial draft

ng@ nfid
1.0 2022-01-20 Michael Shih Official release

gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 2
 CONFIDENTIAL C
Use
Outline

z.c al
tws enti
om
 Hardware-QoS (HQoS)


ng@ nfid
HQoS Rate Limit Example
 QDMA Debug Command

gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 3
 CONFIDENTIAL C
Use
z.c al
tws enti
om
ng@ nfid
Hardware-QoS (HQoS)

gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 4
 CONFIDENTIAL C
Use
HQoS Architecture

z.c al
tws enti
om
DRAM Max-Min Strict-Priority Port
Classifier
Queue Rate Control or WFQ Rate Limit Packets
Q0 bps

ng@ nfid
sch0
…
Q3 bps

gyo Co
Q4 bps

Drop Scheme (FFA/RED)

sch1
…
QDMA Q7 bps
QDMA WAN

din Tek
PPE RX
TX GMAC
Q8 bps

sch2
…
ang edia Q11 bps

Q12 bps
…

sch3
Q15 bps
M

Frame Engine

The mapping of packet-to-queue and queue-to-scheduler are configurable.

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 5
 CONFIDENTIAL C
Use
HNAT Enabled : LAN  WAN

z.c al
tws enti
om
SW NAT
br-lan CPU

ng@ nfid
ra0 eth0 eth1

R-BUS

gyo Co
PDMA QDMA

din Tek
Modify packets
PPE ang edia PSE
NAT info

GMAC1 GMAC2
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved.
 CONFIDENTIAL C
Use
HNAT + HQoS Enabled : LAN -> WAN

z.c al
tws enti
om
SW NAT
br-lan CPU

ng@ nfid
ra0 eth0 eth1

R-BUS

gyo Co
PDMA QDMA

din Tek
Modify packets
PPE ang edia PSE
NAT info

GMAC1 GMAC2
M

entry->ipv4_hnapt.iblk2.fqos=1 indicates enabling QoS scheduling after BINDING

entry->ipv4_hnapt.iblk2.qid=x indicates which QoS queue to be in

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 7
 CONFIDENTIAL C
Use
Panther HQoS Specification

z.c al
tws enti
• 128 physical queues

om
• 4 schedulers

ng@ nfid
• Queue scheduling policy
– Strict-Priority (SP)

gyo Co
- For the queue with higher priority, its min rate is satisfied first.
- For the queue with lower priority, its min rate is satisfied later.

din Tek
- Queue number as the priority, e.g. Queue 0 > Queue 1 > Queue 2…
– Weighted Round-Robin (WRR) ang edia
- After the min rate of each queue is satisfied, the remaining resource is arranged according to the weighting
while not exceed the max rate of each queue.
- Weight as the priority.
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 8
 CONFIDENTIAL C
Use
HQoS Codebase

z.c al
tws enti
• Directory

om
– kernel/drivers/net/ethernet/mediatek/mtk_hnat/

ng@ nfid
• Dependency between Ethernet & HQoS

gyo Co
– Search “CONFIG_NET_MEDIATEK_HW_QOS”
– At eth Tx path: fill skb->mark into TXD

din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 9
 CONFIDENTIAL C
Use
Ways to Enable HQoS Feature (1)

z.c al
tws enti
• make kernel_menuconfig → enable NET_MEDIATEK_HW_QOS

om
ng@ nfid
gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 10
 CONFIDENTIAL C
Use
Ways to Enable HQoS Feature (2)

z.c al
tws enti
• make menuconfig → enable iptables & iptables-mod-ipopt

om
ng@ nfid
gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 11
 CONFIDENTIAL C
Use
Ways to Enable HQoS Feature (3)

z.c al
tws enti
• make menuconfig → enable ebtables & ebtables-ipv4

om
ng@ nfid
gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 12
 CONFIDENTIAL C
Use
Ways to Enable HQoS Feature (4)

z.c al
tws enti
• Enable HQoS

om
– Enable HQoS :

ng@ nfid
CMD> echo 1 > /sys/kernel/debug/hnat/qos_toggle
– Disable HQoS :
CMD> echo 0 > /sys/kernel/debug/hnat/qos_toggle

gyo Co
• Configure HQoS

din Tek
– Edit /etc/config/mtkhnat
– Active Settings:
CMD> sh /sbin/mtkhnat ang edia
• Configure Queueing Policy
– Use iptables command to mark skb
M

– For example: iptables -t mangle -A FORWARD -d 192.168.1.6 -j MARK --set-mark 1

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 13
 CONFIDENTIAL C
Use
HQoS Configurations: Rate Limit

z.c al
tws enti
• /etc/config/mtkhnat.config

om
HW QoS ON/OFF (1:ON, 0:OFF )

txq_num (128 only for Panther)

ng@ nfid
Scheduling policy (‘wrr’ or ‘sp’)
sch0 rate limit (unit: Kbps)

gyo Co
sch1 rate limit (unit: Kbps)

Queue Id

din Tek
Percentage of min rate limit
Percentage of max rate limit
ang edia Weight for queue schedule
Buffer reserved for HW/SW path

The weighted value for TX queue

M
0: Weight value = 16
1: Weight value = 1
2: Weight value = 2
n: Weight value = n
15: Weight value = 15

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 14
 CONFIDENTIAL C
Use
HQoS Configurations: SCH-Q Mapping

z.c al
tws enti
• /sbin/mtkhnat

om
• For example: change Queue_id=1 mapping to Scheduler=1

ng@ nfid
– CMD> echo ${sch_ebl} ${sch_policy} ${sch_bw} > /sys/kernel/debug/hnat/qdma_sch1
- sch_ebl : whether or not enable scheduler

gyo Co
- sch_policy : decide scheduling policy among “sp” or “wrr”
- sch_bw : scheduler bandwidth

din Tek
– CMD> echo 1 ${queue_minebl} ${minrate} ${queue_maxebl} ${maxrate} ${queue_weight}
${queue_resv} > /sys/kernel/debug/hnat/qdma_txq1
- queue_minebl : whether or not enable min rate limit function
ang edia
- queue_maxebl : whether or not enable max rate limit function
M

Scheduler ID Queue ID
MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 15
 CONFIDENTIAL C
Use
Queue Selection via skb->mark

z.c al
tws enti
• Customers can use iptables to direct packets into specified queue.

om
– entry->ipv4_hnapt.iblk2.qid = skb->mark

ng@ nfid
– iptables -t mangle -A FORWARD -p tcp -j MARK --set-mark x skb->mark Queue
0 0
QDMA queue (Q0~Q63) 1 1

gyo Co
..... 2 2
0 1 2 61 62 63
3 3
4 4

din Tek
5 5
SCH0 6 6

...
ang edia QDMA queue (Q64~Q127)
111 111
122 122
123 123
64 65 66 ..... 125 126 127
124 124
M
125 125
126 126
SCH1
127 127

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 16
 CONFIDENTIAL C
Use
z.c al
tws enti
om
ng@ nfid
HQoS Rate Limit Example

gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 17
 CONFIDENTIAL C
Use
HQoS Rate Limit Example (1)

z.c al
tws enti
om
ng@ nfid
Client 1
IP: 10.10.10.3

gyo Co
Server
IP: 192.168.1.33

din Tek
Upstream sch0 = 30Mbps
Queue id = 1, min = 10% , max = 20%
Client 2 Queue id = 6, min = 20% , max = 40%
IP: 10.10.10.22 ang edia Weight = 7
Resv = 4

skb->mark settings
iptables -t mangle -A FORWARD -p tcp -j MARK --set-mark 1 // for eth->eth
M

iptables -t mangle -A FORWARD -p udp -j MARK --set-mark 6

ebtables -t broute -A BROUTING -p ipv4 --ip-src 10.10.10.3 -j mark --mark-set 1 // for wlan->eth
ebtables -t broute -A BROUTING -p ipv4 --ip-src 10.10.10.22 -j mark --mark-set 6

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 18
 CONFIDENTIAL C
Use
HQoS Rate Limit Example (2)

z.c al
tws enti
om
Upstream Limit * Max Rate = 30 * 0.2 = 6 Mbps

Client 1

ng@ nfid
Queue id = 1

gyo Co
din Tek
Upstream Limit * Max Rate = 30 * 0.4 = 12 Mbps

Client 2
ang edia
Queue id = 6
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 19
 CONFIDENTIAL C
Use
z.c al
tws enti
om
ng@ nfid
QDMA Debug Command

gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 20
 CONFIDENTIAL C
Use
QDMA Debug Command (1)

z.c al
tws enti
• Description:

om
– show specific QDMA scheduler info

ng@ nfid
• Command format:
– cat /sys/kernel/debug/hnat/qdma_sch[0~1]

gyo Co
• Example: (show QDMA sch0 info) EN: Rate limit enable or not
Scheduling:
WRR: Use Weighted Round-Robin strategy to select the physical

din Tek
– cat /sys/kernel/debug/hnat/qdma_sch0
queue for the rate between MN and MAX rate.
SP: Use Strict Priority strategy to Select the physical queue for
ang edia the rate between MN and MAX rate.
MAX: Max rate limit for sch0 or sch1
Queue#: QDMA queue in the scheduler
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 21
 CONFIDENTIAL C
Use
QDMA Debug Command (2)

z.c al
tws enti
• Description:

om
– show specific QDMA queue info

ng@ nfid
• Command format:
– cat /sys/kernel/debug/hnat/qdma_txq[0~15]

gyo Co
• Example: (show QDMA txq6 info)

din Tek
– cat /sys/kernel/debug/hnat/qdma_txq6

ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved. 22
 Use
z.c al
tws enti
om
ng@ nfid
gyo Co
din Tek
MediaTek Proprietary and Confidential

© 2021 MediaTek Inc. All rights reserved. The term “MediaTek” refers to MediaTek Inc. and/or its affiliates.
ang edia
This document has been prepared solely for informational purposes. The content herein is made available to a restricted number of clients or partners, for internal
use, pursuant to a license agreement or any other applicable agreement and subject to this notice. THIS DOCUMENT AND ANY ORAL INFORMATION PROVIDED BY
MEDIATEK IN CONNECTION WITH THIS DOCUMENT (COLLECTIVELY THIS “DOCUMENT”), IF ANY, ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND,
WHETHER EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE. MEDIATEK DOES NOT WARRANT OR MAKE ANY REPRESENTATIONS OR GUARANTEE REGARDING THE
M
USE OR THE RESULT OF THE USE OF THIS DOCUMENT IN TERMS OF CORRECTNESS, ACCURACY, TIMELINESS, RELIABILITY, OR OTHERWISE. MEDIATEK SPECIFICALLY
DISCLAIMS ALL WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT AND FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTIES ARISING OUT OF
COURSE OF PERFORMANCE, COURSE OF DEALING OR USAGE OF TRADE. This Document must be held in strict confidence and may not be communicated,
reproduced, distributed or disclosed to any third party or to any other person, or being referred to publicly, in whole or in part at any time except with MediaTek’s
prior written consent, which MediaTek reserves the right to deny for any reason. You agree to indemnify MediaTek for any loss or damages suffered by MediaTek
for your unauthorized use or disclosure of this Document, in whole or in part. If you are not the intended recipient of this document, please delete and destroy all
copies immediately.
 CONFIDENTIAL C
Use
z.c al
tws enti
om
ng@ nfid
gyo Co
din Tek
ang edia
M

MediaTek Proprietary and Confidential. © 2021 MediaTek Inc. All rights reserved.