Network Working Group M. Chadalapaka Request for Comments: 3783 R. Elliott Category: Informational Hewlett-Packard Co. May 2004 Small Computer Systems Interface (SCSI)
Command Ordering Considerations with iSCSI
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Rerved. Abstract
Internet Small Computer Systems Interface (iSCSI) is a Small Computer Systems Interface (SCSI) transport protocol designed to run on top of TCP. The iSCSI ssion abstraction is equivalent to the classic SCSI "I_T nexus", which reprents the logical relationship between an
Initiator and a Target (I and T) required in order to communicate via the SCSI family of protocols. The iSCSI ssion provides an ordered command delivery from the SCSI initiator to the SCSI target. This
document goes into the design considerations that led to the iSCSI
ssion model as it is defined today, relates the SCSI command
ordering features defined in T10 specifications to the iSCSI
concepts, and finally provides guidance to system designers on how
true command ordering solutions can be built bad on iSCSI.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Definitions and Acronyms . . . . . . . . . . . . . . . . . . . 3 2.1. Definitions. . . . . . . . . . . . . . . . . . . . . . . 3
2.2. Acronyms . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Overview of the iSCSI Protocol . . . . . . . . . . . . . . . . 4 3.1. Protocol Mapping Description . . . . . . . . . . . . . . 4 3.2. The I_T Nexus Model. . . . . . . . . . . . . . . . . . . 5 3.3. Ordered Command Delivery . . . . . . . . . . . . . . . . 6 3.3.1. Questions. . . . . . . . . . . . . . . . . . . . 6 3.3.2. The Session Guarantee. . . . . . . . . . . . . . 6 3.3.3. Ordering Onus. . . . . . . . . . . . . . . . . . 7 3.3.
4. Design Intent. . . . . . . . . . . . . . . . . . 7 Chadalapaka & Elliott Informational [Page 1]
4. The Command Ordering Scenario. . . . . . . . . . . . . . . . . 8 4.1. SCSI Layer . . . . . . . . . . . . . . . . . . . . . . . 8 4.1.1. Command Reference Number (CRN) . . . . . . . . . 8 4.1.2. Task Attributes. . . . . . . . . . . . . . . . . 8 4.1.3. Auto Contingent Allegiance (ACA) . . . . . . . . 8 4.1.4. UA Interlock . . . . . . . . . . . . . . . . . . 9
4.2. iSCSI Layer. . . . . . . . . . . . . . . . . . . . . . . 9
5. Connection Failure Considerations. . . . . . . . . . . . . . . 9
6. Command Ordering System Considerations . . . . . . . . . . . . 10
7. Rervation Considerations . . . . . . . . . . . . . . . . . . 11
8. Security Considerations. . . . . . . . . . . . . . . . . . . . 12
9. References and Bibliography. . . . . . . . . . . . . . . . . . 12 9.1. Normative References.. . . . . . . . . . . . . . . . . . 12
9.2. Informative References . . . . . . . . . . . . . . . . . 12
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12
11. Authors’ Address . . . . . . . . . . . . . . . . . . . . . . 13
12. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 14 1. Introduction
iSCSI is a SCSI transport protocol ([iSCSI]) designed to enable
running SCSI application protocols on TCP/IP networks, including
potentially the Internet. Given the size and scope of the Internet, iSCSI thus enables some exciting new SCSI applications. Potential
new application areas for exploiting iSCSI’s value include the
following:
a) Larger (diameter) Storage Area Networks (SANs) than had been
possible until now
b) Asynchronous remote mirroring
c) Remote tape vaulting
Each of the applications takes advantage of the practically
unlimited geographical distance that iSCSI enables between a SCSI
initiator and a SCSI target. In each of the cas, becau of the long delays involved, there is a very high incentive for the
initiator to stream SCSI commands back-to-back without waiting for
the SCSI status of previous commands. Command streaming may be
employed primarily by two class of applications - while one class
may not particularly care about ordered command execution, the other class does rely on ordered command execution (i.e. there is an
application-level dependency on the ordering among SCSI commands).
As an example, cas b) and c) listed earlier clearly require ordered command execution. A mirroring application does not want the writes to be committed out of order on the remote SCSI target, so as to Chadalapaka & Elliott Informational [Page 2]服装搭配学校
prerve the transactional integrity of the data on that target. To summarize, SCSI command streaming, when coupled with the guarantee of ordered command execution on the SCSI target, is extremely valuable
for a critical class of applications in long-latency networks.
hook是什么意思
This document reviews the various protocol considerations in
designing storage solutions that employ SCSI command ordering. This document also analyzes and explains the design intent of [iSCSI] with respect to command ordering.
2. Definitions and Acronyms
2.1. Definitions
- I_T nexus: [SAM2] defines the I_T nexus as a relationship between a SCSI initiator port and a SCSI target port. [iSCSI] defines an iSCSI ssion as the iSCSI reprentation of an I_T nexus. In the iSCSI context, the I_T nexus (i.e. the iSCSI ssion) is a
fundamentumrelationship between an iSCSI initiator’s end of the ssion (SCSI Initiator Port) and the iSCSI target’s Portal Group (SCSI Target
Port).
- PDU (Protocol Data Unit): An iSCSI initiator and iSCSI target
communicate using iSCSI protocol messages. The messages are
called "iSCSI protocol data units" (iSCSI PDUs).
- SCSI device: A SCSI device is an entity that contains one or more SCSI ports that are connected to a rvice delivery subsystem and supports SCSI application protocols. In the iSCSI context, the
SCSI Device is the component within an iSCSI Node that provides
2014ama全美音乐奖the SCSI functionality. The SCSI Device Name is defined to be the iSCSI Name of the node.
- Session: A group of logically related iSCSI connections that link an initiator with a target form a ssion (equivalent to a SCSI
I-T nexus). The number of participating iSCSI connections within an iSCSI ssion may vary over time. The multiplicity of
connections at the iSCSI level is completely hidden for the SCSI
layer - each SCSI port in an I_T nexus es only one peer SCSI
port across all the connections of a ssion.
Chadalapaka & Elliott Informational [Page 3]
2.2. Acronyms
Acronym Definition
--------------------------------------------------------------
ACA Auto Contingent Allegiance
ASC Additional Sen Code
ASCQ Additional Sen Code Qualifier
CRN Command Reference Number
IETF Internet Engineering Task Force
ISID Initiator Session Identifier
ITT Initiator Task Tag
LU Logical Unit
LUN Logical Unit Number
NIC Network Interface Card
PDU Protocol Data Unit
TMF Task Management Function
TSIH Target Session Identifying Handle
SAN Storage Area Network
SCSI Small Computer Systems Interface
TCP Transmission Control Protocol
UA Unit Attention
WG Working Group
3. Overview of the iSCSI Protocol
3.1. Protocol Mapping Description
The iSCSI protocol is a mapping of the SCSI remote procedure
invocation model (e [SAM2]) over the TCP protocol.
SCSI’s notion of a task maps to an iSCSI task. Each iSCSI task is
uniquely identified within that I_T nexus by a 32-bit unique
identifier called Initiator Task Tag (ITT). The ITT is both an iSCSI identifier of the task and a classic SCSI task tag.
SCSI commands from the initiator to the target are carried in iSCSI
requests called SCSI Command PDUs. SCSI status back to the initiator is carried in iSCSI respons called SCSI Respon PDUs. SCSI Data- out from the initiator to the target is carried in SCSI Data-Out
utmPDUs, and the SCSI Data-in back to the initiator is carried in SCSI
Data-in PDUs.
Chadalapaka & Elliott Informational [Page 4]
3.2. The I_T Nexus Model
In the iSCSI model, the SCSI I_T nexus maps directly to the iSCSI
ssion, which is an iSCSI protocol abstraction spanning one or more TCP connections. The iSCSI protocol defines the mantics in order
to realize one logical flow of bidirectional communication on the I_T nexus, potentially spanning multiple TCP connections (as many as
solon2^16). The multiplicity of iSCSI connections is thus completely
contained at the iSCSI layer, while the SCSI layer is prented with a single I_T nexus, even in a multi-connection ssion. A ssion
between a pair of given iSCSI nodes is identified by the ssion
identifier (SSID) and each connection within a given ssion is英语谚语翻译
uniquely identified by a connection identifier (CID) in iSCSI. The
SSID itlf has two components - Initiator Session Identifier (ISID) and a Target Session Identifying Handler (TSIH) - each identifying
one end of the same ssion.
There are four crucial functional facets of iSCSI that together
prent this single logical flow abstraction to the SCSI layer, even with an iSCSI ssion spanning across multiple iSCSI connections.
a) Ordered command delivery: A quence of SCSI commands that is
striped across all the connections in the ssion is
"reordered" by the target iSCSI layer into an identical
quence bad on a Command Sequence Number (CmdSN) that is
unique across the ssion. The goal is to achieve bandwidth
aggregation from multiple TCP connections, but to still make it appear to the target SCSI layer as if all the commands had
travelled in one flow.
b) Connection allegiance: All the PDU exchanges for a SCSI
Command, up to and including the SCSI Respon PDU for the
Command, are required to flow on the same iSCSI connection at
any given time. This again is intended to hide the multi-
connection nature of a ssion becau the SCSI layer on either side will never e the PDU contents out of order (e.g., status cannot bypass read data for an initiator).
c) Task t management function handling: [iSCSI] specifies an
ordered quence of steps for the iSCSI layer on the SCSI
target in handling the two SCSI task management functions
(TMFs) that manage SCSI task ts. The two TMFs are ABORT TASK SET that aborts all active tasks in a ssion, and CLEAR TASK
SET that clears the tasks in the task t. The goal of the
quence of steps is to guarantee that the initiator receives
the SCSI Respon PDUs of all unaffected tasks before the TMF
Respon itlf arrives, regardless of the number of
connections in the iSCSI ssion. This operational model is Chadalapaka & Elliott Informational [Page 5]
again intended to prerve the single flow abstraction to the
SCSI layer.
d) Immediate task management function handling: Even when a TMF
request is marked as "immediate" (i.e. only has a position in
the command stream, but does not consume a CmdSN), [iSCSI]
defines mantics that require the target iSCSI layer to ensure that the TMF request is executed as if the commands and the TMF request were all flowing on a single logical channel. This
ensures that the TMF request will act on tasks that it was
meant to manage.
The following ctions will analyze the "Ordered command delivery"
aspect in more detail, since command ordering is the focus of this
bullfrogdocument.
3.3. Ordered Command Delivery
3.3.1. Questions
A couple of important questions related to iSCSI command ordering
were considered early on in the design of the iSCSI protocol. The
questions were:
a) What should be the command ordering behavior required of iSCSI implementations in the prence of transport errors, such as
errors that corrupt the data in a fashion that is not detected by the TCP checksum (e.g., two offtting bit flips in the same bit position), but is detected by the iSCSI CRC digest?
b) Should [iSCSI] require both initiators and targets to u
ordered command delivery?
Since the answers to the questions are critical to the
understanding of the ordering behavior required by the iSCSI
protocol, the following sub-ctions consider them in more detail.
3.3.2. The Session Guarantee
differentfrom
The final disposition of question a) in ction 3.3.1 was reflected
ts什么意思
in [RFC3347], "iSCSI MUST specify strictly ordered delivery of SCSI
commands over an iSCSI ssion between an initiator/target pair, even in the prence of transport errors." Stated differently, an iSCSI
digest failure, or an iSCSI connection termination, must not cau
the iSCSI layer on a target to allow executing the commands in an
order different from that intended (as indicated by the CmdSN order) by the initiator. This design choice is enormously helpful in
building storage systems and solutions that can now always assume Chadalapaka & Elliott Informational [Page 6]
