Next Patent: Technique for implementing chipkill in a memory system
Next Patent: Technique for implementing chipkill in a memory system
[0001] The following commonly owned application is hereby incorporated by reference for all purposes:
[0002] U.S. patent application Ser. No.______, filed concurrently herewith, entitled “Technique for Implementing Chipkill in a Memory System” by Sompong P. Olarig.
[0003] 1. Field of the Invention
[0004] This invention relates generally to computer systems and, more particularly, to error handling in a memory system implementing X8 memory devices.
[0005] 2. Background of the Related Art
[0006] This section is intended to introduce the reader to various aspects of art which may be related to various aspects of the present invention which are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background infonnation to facilitate a better understanding of the various aspects of the present invention. Accordingly, it should be understood that these statements are to le read in this light, and not as admissions of prior art.
[0007] Computer systems, such as the personal computers and servers, rely on microprocessors, associated chip sets, and memory chips to perform most of their processing functions. In contrast to the dramatic improvements of the processing portions of a computer system, the mass storage portion of a computer system has experienced only modest growth in speed and reliability. As a result, computer systems fail to capitalize fully on the increased speed of the improving processing systems due to the dramatically inferior capabilities of the mass data storage devices coupled to the systems.
[0008] While the speed of these mass storage devices, such as magnetic disk drives, has not improved much in recent years, the size of such disk drives has become smaller while maintaining the same or greater storage capacity. Furthermore, such disk drives have become less expensive. To capitalize on these benefits, it was recognized that a high capacity data storage system could be realized by organizing multiple small disk drives into an array of drives. However, it was further recognized that large numbers of smaller disk drives dramatically increased the chance of a disk drive failure which, in turn, increases the risk of data loss. Accordingly, this problem has been addressed by including redundancy in the disk drive arrays so that data lost on any failed disk drive can be reconstructed through the redundant information stored on the other disk drives. This technology has been commonly referred to as “redundant arrays of inexpensive disks” (RAID).
[0009] To date, at least five different levels of RAID have been introduced. The first RAID level (“RAID Level 1”) utilizes mirrored devices. In other words, data is written identically to at least two disks. Thus, if one disk fails, the data can be retrieved from one of the other disks. Of course, a RAID Level 1 system requires the cost of an additional disk without increasing overall memory capacity in exchange for decreased likelihood of data loss. The second level of RAID (“RAID Level 2”) implements an error code correction or “ECC” (also called “error check and correct”) scheme where additional check disks are provided to detect single errors, identify the failed disk, and correct the disk with the error. The third level RAID system (“RAID Level 3”) stripes data at a byte-level across several drives and stores parity data in one drive. RAID Level 3 systems generally use hardware support to efficiently facilitate the byte-level striping. The fourth level of RAID (“RAID Level 4”) stripes data at a block-level across several drives, with parity stored on one drive. The parity information allows recovery from the failure of any single drive. The performance of a RAID Level 4 array is good for read requests. Writes, however, may require that parity data be updated each time. This slows small random writes, in particular, though large writes or sequential writes may be comparably faster. Because only one drive in the array stores redundant data, the cost per megabyte of a RAID Level 4 system may be fairly low. Finally, a level 5 RAID system -(“RAID Level 5”) provides block-level memory striping where data and parity information are distributed in some form throughout the disk drives in the array. Advantageously, RAID Level 5 systems may increase the processing speed of small write requests in a multi-processor system since the parity disk does not become a system bottleneck.
[0010] The implementation of data redundancy, such as in the RAID schemes discussed above, provides fault tolerant computer systems wherein the system may still operate without data loss, even if one drive fails. This is contrasted to a disk drive array in a non-fault tolerant system where the entire system is considered to have failed if any one of the drives fails. Of course, it should be appreciated that each RAID scheme necessarily trades some overall storage capacity and additional expense in favor of fault tolerant capability. Thus, RAID systems are primarily found in computers performing mission critical functions where failures are not easily tolerated. Such functions may include, for example, a network server, a web server, a communication server, etc. One of the primary advantages of a fault tolerant mass data storage system is that it permits the system to operate even in the presence of errors that would othervise cause the system to malfunction. As discussed previously, this is particularly important in critical systems where downtime may cause relatively major economic repercussions.
[0011] As with disk arrays, memory devices may be arranged to form memory arrays. For instance, a number of Dynamic Random Access Memory (DRAM) devices may be configured to form a single memory module, such as a Dual Inline Memory Module (DIMM). The memory chips on each DIMM are typically selected from one or more DRAM technologies, such as synchronous DRAM, double data rate SDRAM, direct-RAM bus, and synclink DRAM, for example. Typically, DIMMs are organized into an X4 (4-bit wide), an X8 (8-bit wide), or larger fashion. In other words, the memory chips on the DIMM are either 4-bits wide, 8-bits wide, 16-bits wide or 32 -bits wide. To produce a 72-bit data word using an X4 memory organization, an exemplary DIMM may include nine 4-bit wide memory chips located on one side of the DIMM and nine 4-bit wide memory chips located on the opposite side of the DIMM. Conversely, to produce a 72-bit data word using an X8 memory organization, an exemplary DIMM may include nine 8-bit wide memory chips located on a single side of the DIMM. The memory modules may be arranged to form memory segments and the memory segments may be combined to form memory arrays. Controlling the access to and from the memory devices as quickly as possible while adhering to layout limitations and maintaining as much fault tolerance as possible is a challenge to system designers.
[0012] One mechanism for improving fault tolerance is to provide a mechanism such as an Error Checking and Correcting (ECC) algorithm. ECC is a data encoding and decoding scheme that uses additional data bits to provide error checking and correcting capabilities. Today's standard ECC algorithms, such as the Intel P6 algorithm, can detect single-bit or multi-bit errors within an X4 memory device. Further, typical ECC algorithms provide for single-bit error correction (SEC). However, typical FCC algoritlms alone may not be able to correct multi-bit errors. Further, while typical ECC algorithms may be able to detect single bit errors in X8 devices, they cannot reliably detect multi-bit errors in X8 devices, much less correct those errors. In fact, approximately 25% of all possible multi-bit errors within an X8 memory device are either undetected or wrongly detected as single-bit errors or “misaliased.” Misaliasing refers to multi-bit error conditions in an X8 (or larger) memory device that defeat standard ECC algorithms such that the multi-bit errors appear to the ECC logic to be either correct data or data with a single-bit correctable error.
[0013] The foregoing and other advantages of the invention will become apparent upon reading the following detailed description and upon reference to the drawings in which:
[0014]
[0015]
[0016]
[0017]
[0018] One or more specific embodiments of the present invention will be described below. In an effort to provide a concise description of these embodiments, not all features of an actual implementation are described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure.
[0019] Turning now to the drawings and referring initially to
[0020] Each of the buses
[0021] As will be appreciated from the discussion herein, the number of memory segments
[0022] Each of the memory controllers
[0023] As will be explained in greater detail below, during a data read operation, the host/data controller
[0024] Each memory controller
[0025] In this embodiment, each of the memory segments
[0026] The memory segments
[0027] Returning to the exemplary system
[0028] As previously discussed, it may be advantageous to provide one or more mechanisms to improve fault tolerance in the memory array
[0029] Current ECC algorithms can detect and correct single-bit errors in X4 memory devices. Disadvantageously, current ECC algorithms cannot reliably detect multi-bit errors in X8 devices, much less correct such errors. The present RAID system, in conjunction with typical ECC algorithms, provides a mechanism for correcting multi-bit errors in X4 devices, as described below. In fact, the presently described system provides a mechanism for overcoming the complete failure of a memory device. “Chipkill” is an industry standard term referring to the ability of a memory system to withstand a complete memory device failure and continue to operate normally. Advantageously, when an error occurs, it effects only a single device. Thus, chipkill eliminates the vast majority of error conditions in a memory system. Disadvantageously, X4 chipkill may not provide sufficient fault tolerance to meet the requirements for all systems.
[0030] Memory devices are gradually transitioning from X4 to X8 devices due to larger capacities per device. However, while DIMMs incorporating X8 devices have become more common place, the ECC algorithm used in industry standard devices has remained the same. The standard ECC algorithm, such as the P6 algorithm defined by Intel, can only provide chipkill detection capabilities for X4 (or narrower) devices. For those systems that incorporate X8 capabilities, it may be advantageous to provide an ECC algorithm capable of reliably detecting multi-bit errors in X8 devices. Further, a mechanism for correcting the multi-bit errors in X8 memory devices would be advantageous. Ihowever, it may also be beneficial to provide a system whose fault tolerance is not degraded if X4 memory devices are incorporated.
[0031] The P6 ECC is an encoding scheme which can correct single data bit errors, and detect multi-bit errors, but only up to four adjacent bits within the same nibble or 4 adjacent bits. When a chipkill greater than 4 bits occurs, the P6 ECC logic may inadvertently correct the wrong data bit. In some cases, the P6 ECC cannot detect some multi-bit errors at all. Therefore, the XOR engine (described with reference to FIGS.
[0032] With the implementation of wider memory devices, such as X8 memory devices, it would be desirable for fault-tolerant memory to be able to detect all possible errors in the X8 memory devices, as well as narrower memory devices, such as X4 memory devices. By implementing a new dual mode ECC parity checking algorithm or “matrix” in conjunction with the presently described RAID memory system, the present system is able to provide chipkill detection and correction capabilities for X8 memory devices. The presently described matrix is simply one exemplary embodiment of an algorithm that is capable of detecting multi-bit errors in an 8-bit (byte) segment. Numerous alternate embodiments of the matrix may be utilized in the present system, as can be appreciated by those skilled in the art.
[0033] The dual mode ECC is a dual-purpose error correcting/checking code. When operating in a X4 mode, it acts as a single bit correcting code and can detect errors in the adjacent 4 bits within the same nibble (similar to P6 ECC). When operating in X8 mode, it acts as an
[0034] Table 1 defines eight bits that define the syndrome for the error correcting code, wherein each row of Table 1 corresponds to a syndrome bit. The eight syndrome bits are created by eight equations defined by Table 1. Each row of the table represents 72-bits that include 64 data bits and 8 ECC bits. As the 72-bits are read, the data bits are combined in a manner defined by the matrix in Appendix 1. For each row of bits, the data residing at each bit location where a logical 1 is illustrated in Table 1, the bit at that location is combined with all other bits in locations corresponding to the logical Is in Table 1. The bits are combined together as logical XORs. Thus, each row of Table 1 defines an equation that is used to combine 64 data bits and 8 check bits to produce a single bit logical result. As indicated, row 1 corresponds to syndrome bit [0], row 2 corresponds to syndrome bit [1], etc. The eight syndrome bits are ordered to produce a single HEX number known as the syndrome that can be interpreted using Table 2, illustrated in Appendix 2. The eight syndrome bits are ordered from syndrome bit [7] to syndrome bit [0] to form the index used in Table 2.
[0035] An exemplary dual-mode ECC syndrome interpretation table is illustrated with reference to Appendix 2, below. All uncorrectable errors (“UNCER”) will be flagged but will not be corrected by the ECC module. That is to say that the error looks like a multi-bit error and cannot be corrected by the ECC code. If no error is detected, the HEX value (i.e. the syndrome) will be “00.” If a single bit error is detected, the HEX value produced by Table 1 will correspond to an error location identifying the single bit error. For example, if Table 1 produces a value of 68 hex, a single bit error was detected at data bit 27 (DB27). If the ECC algorithm is operating in a X4 or normal ECC mode, the single bit error will be corrected by the ECC code. If the ECC algorithm is operating in a X8 mode, any memory error (single or multi-bit) will not be corrected by the ECC code. In fact, if the ECC algorithm is operating in a X8 mode, all syndrome codes except 00 hex are considered tncorrectable errors that will not be corrected by the ECC algorithm. This is because in X8 mode some MBEs will map to SBE correctable errors (i.e. the errors are miscorrected or “misaliased”) Check bits, like data bits, can be in error. If the HEX value is 01, then the bit error is the ECC code bit CB0.
[0036] Thus, the dual-mode ECC algorithm may be implemented to effectively handle errors in X8 memory devices. The present system provides a memory array with X8 chipkill by implementing the dual-mode ECC algorithm in conjunction with the RAID logic to provide a new level of memory fault tolerance. To implement X8 memory devices in the memory array
[0037] The manner in which an exemplary “4+1” RAID architecture functions will now be explained with reference to
[0038] When operating in X8 memory mode (i.e. the meniory system
[0039] Once the memory controllers
[0040] Each ECC module
[0041] The data controller
[0042] An introduction to an exemplary embodiment of the functionality of the data striping and correction techniques as implemented during a read operation are briefly described with reference to
[0043] Turning now to
[0044] DATA WORD 1: 1 0 1 1
[0045] DATA WORD 2: 0 0 1 0
[0046] DATA WORD 3: 1 0 0 1
[0047] DATA WORD 4: 0 1 1 1
[0048] A parity word can be either even or odd. To create an even parity word, common bits are simply added together. If the sum of the common bits is odd, a “1” is placed in the common bit location of the parity word. Conversely, if the sum of the bits is even, a zero is placed in the common bit location of the parity word. In the present example, the bits may be summed as follows:
[0049] DATA WORD 1: 1 0 1 1
[0050] DATA WORD 2: 0 0 1 0
[0051] DATA WORD 3: 1 0 0 1
[0052] DATA WORD 4: 0 1 1 1
[0053] {overscore (NUMBER OF 1's: 2 1)} 3 3
[0054] PARITY WORD: 0 1 1 1
[0055] When summed with the four exemplary data words, the parity word 0111 will provide an even number of active bits (or “1's”) in every common bit. This parity word can be used to re-create any of the data words (1-4) if an error is detected in one of the data words as further explained with reference to
[0056]
[0057] Similarly, if the memory controller
[0058] Similarly, the XOR engine
[0059] Returning to
[0060] As described above, one side-effect of changing to the presently described approach to error correction is that a class of error that previously was corrected by the ECC SEC logic, becomes an uncorrectable error condition. Recall that when the system is operating in X4 memory mode, the dual-mode ECC module
[0061] Conversely, in X8 memory mode single-bit error correction is turned off. By turning off single-bit error correction, the above-mentioned misaliasing problem with X8 devices is eliminated and the dual-mode ECC algorithm provides X8 chipkill detection. In this mode of operation, all errors, including single-bit errors, are corrected by the XOR engine
[0062] When operating in X8 mode, one solution to the SBE-SBE condition is a trade-off. If one single-bit error occurs, the XOR engine
[0063] To implement this solution in X8 mode, an XOR mux controller
[0064] While the invention may be susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, it should be understood that the invention is not intended to be limited to the particular forms disclosed. Rather, the invention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the invention as defined by the following appended claims.
[0065]
TABLE 1 Dual Mode ECC Parity-Check Matrix Nibble# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Syndrome [0] 1000 0000 1000 0000 1000 1111 1000 0000 1000 1111 1000 1111 0100 1000 0001 1100 1100 1001 Syndrome [1] 0100 0000 0100 1111 0100 0000 0100 1111 0100 0000 0100 1111 0100 1001 0011 1000 0100 1100 Syndrome [2] 0010 1111 0010 0000 0010 0000 0010 1111 0010 1111 0010 0000 0110 1100 0010 1100 0100 1000 Syndrome [3] 0001 1111 0001 1111 0001 1111 0001 0000 0001 0000 0001 0000 1100 1000 0011 0100 0110 1000 Syndrome [4] 0000 1000 0000 1000 1111 1000 0000 1000 1111 1000 1111 1000 1000 0100 1100 0001 1001 1100 Syndrome [5] 0000 0100 1111 0100 0000 0100 1111 0100 0000 0100 1111 0100 1001 0100 1000 0011 1100 0100 Syndrome [6] 1111 0010 0000 0010 0000 0010 1111 0010 1111 0010 0000 0010 1100 0110 1100 0010 1000 0100 Syndrome [7] 1111 0001 1111 0001 1111 0001 0000 0001 0000 0001 0000 0001 1000 1100 0100 0011 1000 0110
[0066]
TABLE 2 Dual Mode ECC Syndrome Interpretation Table 00 No Error 20 CB5 40 CB6 60 UNCER 80 CB7 A0 UNCER C0 UNCER E0 DB58 01 CB0 21 UNCER 41 UNCER 61 DB24 81 UNCER A1 DB8 C1 DB0 E1 UNCER 02 CB1 22 UNCER 42 UNCER 62 DB25 82 UNCER A2 DB9 C2 DB1 E2 UNCER 03 UNCER 23 DB45 43 DB46 63 UNCER 83 DB47 A3 UNCER C3 UNCER E3 UNCER 04 CB2 24 UNCER 44 UNCER 64 DB26 84 UNCER A4 DB10 C4 DB2 E4 UNCER 05 UNCER 25 DB37 45 DB38 65 UNCER 85 DB39 A5 UNCER C5 UNCER E5 UNCER 06 UNCER 26 DB29 46 DB30 66 UNCER 86 DB31 A6 UNCER C6 UNCER E6 UNCER 07 DB56 27 UNCER 47 UNCER 67 UNCER 87 UNCER A7 UNCER C7 UNCER E7 UNCER 08 CB3 28 UNCER 48 UNCER 68 DB27 88 UNCER A8 DB11 C8 DB3 E8 UNCER 09 UNCER 29 DB21 49 DB22 69 UNCER 89 DB23 A9 UNCER C9 UNCER E9 UNCER 0A UNCER 2A DB13 4A DB14 6A UNCER 8A DB15 AA UNCER CA UNCER EA UNCER 0B DB55 2B UNCER 4B UNCER 6B UNCER 8B UNCER AB UNCER CB UNCER EB UNCER 0C UNCER 2C DB5 4C DB6 6C UNCER 8C DB7 AC UNCER CC UNCER EC UNCER 0D DB57 2D UNCER 4D UNCER 6D UNCER 8D UNCER AD UNCER CD UNCER ED UNCER 0E DB54 2E UNCER 4E UNCER 6E UNCER 8E UNCER AE UNCER CE UNCER EE UNCER 0F UNCER 2F DB61 4F DB49 6F UNCER 8F DB50 AF UNCER CF UNCER EF UNCER 10 CB4 30 UNCER 50 UNCER 70 DB52 90 UNCER B0 DB59 D0 DB53 F0 UNCER 11 UNCER 31 DB40 51 DB32 71 UNCER 91 DB16 B1 UNCER D1 UNCER F1 DB60 12 UNCER 32 DB41 52 DB33 72 UNCER 92 DB17 B2 UNCER D2 UNCER F2 DB63 13 DB44 33 UNCER 53 UNCER 73 UNCER 93 UNCER B3 UNCER D3 UNCER F3 UNCER 14 UNCER 34 DB42 54 DB34 74 UNCER 94 DB18 B4 UNCER D4 UNCER F4 DB51 15 DB36 35 UNCER 55 UNCER 75 UNCER 95 UNCER B5 UNCER D5 UNCER F5 UNCER 16 DB28 36 UNCER 56 UNCER 76 UNCER 96 UNCER B6 UNCER D6 UNCER F6 UNCER 17 UNCER 37 UNCER 57 UNCER 77 UNCER 97 UNCER B7 UNCER D7 UNCER F7 UNCER 18 UNCER 38 DB43 58 DB35 78 UNCER 98 DB19 B8 UNCER D8 UNCER F8 DB48 19 DB20 39 UNCER 59 UNCER 79 UNCER 99 UNCER B9 UNCER D9 UNCER F9 UNCER 1A DB12 3A UNCER 5A UNCER 7A UNCER 9A UNCER BA UNCER DA UNCER FA UNCER 1B UNCER 3B UNCER 5B UNCER 7B UNCER 9B UNCER BB UNCER DB UNCER FB UNCER 1C DB4 3C UNCER 5C UNCER 7C UNCER 9C UNCER BC UNCER DC UNCER FC UNCER 1D UNCER 3D UNCER 5D UNCER 7D UNCER 9D UNCER BD UNCER DD UNCER FD UNCER 1E UNCER 3E UNCER 5E UNCER 7E UNCER 9E UNCER BE UNCER DE UNCER FE UNCER 1F DB62 3F UNCER 5F UNCER 7F UNCER 9F UNCER BF UNCER DF UNCER FF UNCER