New HP Drivers...

hi grog,

let's take small steps, and learn in the process.
It is what I still do.
I learned to make new mistakes every day.

If you really want to start making additions, you better read this.
-----------------------------------------------------
Drivers and the Mass storage Driverpack;

glossary

1; driver.INF
Information the Operating System requires to support hardware; including registry entries and files to copy.

2; driver.CAT
Security catalog. A *.CAT file can be WHQL signed but this is not always the case.

3; driver.SYS
System level support for hardware (the driver!)

4; HWID
Hardware identity, vendor and device.
This is the number the device reports to plug and play to match with a driver file.

5; driverpack.INI
This file is used to identify exceptions and special cases, such as, this driver only has windows 2003 support.

6; generic HWID (VendorID&deviceID, NO SUB.)
Example PCI\VEN_13C1&DEV_1002
Example PCI\VEN_1191&DEV_0006

7; SUB Specified HWID
Subversions of a chip can exist. Either proprietary made for an OEM, or an improved mask was used at the Fabrication plant. They will have SUB_SYS and sometimes Revision info after the generic ID.
Example PCI\VEN_13C1&DEV_1002&SUBSYS_100213C1
Example PCI\VEN_1191&DEV_0006&SUBSYS_00061191&REV_01

8; mask
Used in etching process at the Plant. A new mask etches different traces and a different chip is built.

9; Original Equipment Manufacturer (OEM)
Large OEMs like DELL, NEC or HP/Compaq often have proprietary chips on the motherboards.

10; Windows Hardware Quality Labs (WHQL)
Drivers that have been submitted and successfully passed the tests at Microsoft will get the WHQL signed status.
WHQL was introduced when XP was in its infancy and you will seldom find very old WHQL signed drivers because of this.
Windows Driver Model (WDM) are most likely good drivers because this standard is still in use.
Unstable drivers do exist. Information and rants about unstable drivers can be found on the internet.
We will attempt to avoid any known unstable drivers.

---------------------------------------------
Next, a reference TXT.
It was written by RuudBoek
---------------------------------------------
Editing the Mass storage driverpack INI.

Q: What does the entry "ms_count = 1" mean?
A: The number next to the "ms_count =" entry represents the amount of *.sys files that are present in the current section and folder that should be used during the slipstream. This entry can also be used to exclude a driver from all OS'es during slipstreaming.
To do this, simply add a zero next to the "ms_count =" entry. For example:

"ms_count = 0"

Q: What does the "ms_1_deviceName" entry mean?
A: The "ms_1_deviceName" entry represents the full driver name (Its friendly name).
One driver can often support multiple hardware devices. For example: "IBM ServeRAID 4H Controller" and "IBM ServeRAID 3H/3L Controller" both share the same driver files. The "ms_1_deviceName" entry enables you to specify one name representing all these different hardware devices. This name is displayed during the loading of the Mass Storage drivers for textmode setup. Since this name has no impact on the proper functioning of the driver itself, it is possible to make up a name by yourself. It is however strongly advised to use a name similar to the name that is used by the driver manufacturer.

Q: What does the "ms_1_tag" entry mean?
A: Most of the time the "ms_1_tag" represents the name of the .sys file included in the driver.
This name is used to create the compressed files which are located in the I386 folder (example: SI3124.SY_).
This compressed file is decompressed during Text Mode* part of setup and loaded into memory so that mass storage media is accessible.

*Textmode setup is the blue screen part at the beginning of the installation and it enables you to partition your hard drive for example.

In some cases there are .sys files present from different driver folders that have the same file names.
They can not be compressed and placed into the I386 folder using the same filename because the resulting compressed files would bear the same filename also. When this occurs a workaround is used. The resulting compressed files get different filenames than the original .sys filename.
The "ms_1_tag" entry enables you to specify unique filenames for filenames that would have otherwise been duplicated during the copy to I386.

Q: Where can I find the "ms_1_sysFile" entry in my driver?
A:The "ms_1_sysFile" entry represents the name of the .sys file which should always be included in your driver. If there is more then one .sys file present in the driverpackage, please bear in mind that only the BUS driver is relevant. Often you will find .sys files which include the following in their names:

filter, filt, fltr, filtr, etc.
For example: xfilt.sys.

Those .sys files are called filters and are NOT bus drivers. They can therefore be ignored.

Often you will also find .sys files which includes the following in their names:
64
For example: videX64.sys

Most of the time that value means that the .sys file was built for the 64-bit version of Windows XP. Since the DriverPacks Mass Storage only supports the 32-bit version of Windows XP, those .sys files can also be ignored.

Q: What does the "ms_1_hwids" entry mean?
A: The "ms_1_hwids" entry represents all the hardware id's that are associated with the driver.

Q: Where can I find the correct hardware id's for my driver?
A: You can usually find the drivers hardware id's inside the txtsetup.oem file.
It is strongly recommended to always use the HWID's inside the txtsetup.oem file when available. If a txtsetup.oem file is not present you will be forced to look inside the .inf files for the driver IDs.

When looking inside a txtsetup.oem file, first Look for a section with the following in its name: HardwareIds.SCSI
For example:
[HardwareIds.SCSI.Si3112_XP]
id = "PCI\VEN_1095&DEV_3112&SUBSYS_31121095", "Si3112"
id = "PCI\VEN_1095&DEV_3112&SUBSYS_34238086", "Si3112"
id = "PCI\VEN_1095&DEV_3112&SUBSYS_311215D9", "Si3112"
id = "PCI\VEN_1095&DEV_3112&SUBSYS_B0021458", "Si3112"

The actual hardwareid's are located in the specified section ([HardwareIds.SCSI.Si3112_XP].
Hardwareid's always start with PCI\... In this example the correct hardwareid's are therefore the following:

PCI\VEN_1095&DEV_3112&SUBSYS_31121095
PCI\VEN_1095&DEV_3112&SUBSYS_34238086
PCI\VEN_1095&DEV_3112&SUBSYS_311215D9
PCI\VEN_1095&DEV_3112&SUBSYS_B0021458

Simple right! Well, there's more.

When no txtsetup.oem file is present you need to look inside the .inf files of the driver.
In this case open the .inf file(s) and look for a section called [Manufacturer] in that section you should be able to find the name of the section where the HWID's can be found.
For example:

[Manufacturer]
%ITE% = ITE.Mfg

In this example, we will thefore find all the hardwareid's in the following section:
[ITE.Mfg]

Sometimes more than one section is specified underneath [Manufacturer]. Bear in mind that if one of those sections include the value 64 that specific section can be normally be ignored. It most probably represents 64-bit Windows driver which is not currently supported.

The hardware id's themselves are located at the end of each line in the specified sections.
For this example:
[ITE.Mfg]
%ITE.DeviceDesc0% = iteraid, PCI\VEN_1283&DEV_8212&SUBSYS_00011283
%ITE.DeviceDesc0% = iteraid, PCI\VEN_1283&DEV_8212&SUBSYS_00000000

Hardwareid's always start with PCI\... In this example the correct hardwareid's are therefore the following:
PCI\VEN_1283&DEV_8212&SUBSYS_00011283
PCI\VEN_1283&DEV_8212&SUBSYS_00000000

Hardware id's should be added to the "ms_1_hwids" entry; all on one line, with no spaces, and each HWID should be separated by a comma, the complete line should also start with double quotes and end with double quotes.
For example:
ms_1_hwids = "PCI\VEN_1283&DEV_8212&SUBSYS_00011283,PCI\VEN_1283&DEV_8212&SUBSYS_00000000"

Q:What does the entry "ms_1_isBusExtender" mean?
A:The "ms_1_isBusExtender" entry specifies wether the driver is a bus extender or not.

Q:How can I find out if my driver is a bus extender?
A:Open the .inf files which should be included in the driver. Look for a line similar to the following:

LoadOrderGroup = System Bus Extender
If that line is present then your driver is a busextender.

Q:What does the entry "ms_1_exc_skipIfOS" mean?
A:The "ms_1_exc_skipIfOS" entry represents the operating system to which the driver(s) will not be slipstreamed.
For example:
ms_1_exc_skipIfOS= "w2k"
In this example the entry will prevent the specified driver from being slipstreamed into Windows 2000.

Q:What does the entry "ms_1_exc_disableIfOS" mean?
A:The "ms_1_exc_skipIfOS" entry represents the operating system in which the driver should not be included during textmode setup.

For example:
Ms_1_exc_disableIfOS= "w2k"

In this example the entry will prevent the specified driver from being loaded into memory during textmode setup when the OS is Windows 2000. After the slipstream completes the driver will be present in txtsetup.sif and dosnet.inf but the entry will be prefixed by a semi-colon which will cause setup to ignore that line and therefore the driver on that line.

Excluding a driver to be loaded during textmode setup can be quite useful. Windows 2000, for example, is very limited in the amount of memory that can be allocated for loading textmode drivers. So we can easily select some rarely used drivers to be "on hold" for text mode but still available to plug and play in windows afterwards. So far we have only needed to do this for windows 2000 and it is technically only supported for windows 2000, however, it is only a matter of time until this limit becomes a factor for XP and eventually 2003.

Q:What does the entry "ms_1_exc_replaceIfOS" mean?
A:The "ms_1_exc_replaceIfOS" entry represents the operating system in which the specified driver should replace an already existing driver.
Some operating systems already contain drivers for textmode setup by default.
For example:
Windows XP contains a driver called "Mylex AcceleRAID 160 Disk Array Controller".
This driver can be found in the I386 folder and it is named dac2w2k.sy_. If you want to overwrite this driver with a newer version you can use the following entry to tell the DPs_Base to overwrite the existing dac2w2k driver:
ms_1_exc_replaceIfOS= "wxp"

When Ruud posted this, the project had not yet gone open source.
The info is now available to you.

=============================

Let us go on about Mass storage Drivers.

First, let us mention something about the security catalog file, the CAT:
When one edits a duplicate driver's INF file to add HWIDs, its CAT is no longer valid for all HWIDs.
The supported devices in INF that do not match devices listed in the CAT are not signed when a HWID match occurs.
Yes! a CAT file actually lists HWIDs. Engineers using DDK can edit it, but after that it needs to be submitted and re-signed. Signed drivers are published as "WHQL signed" at manufacturers download sites. We do not use DDK since we would not be successful in getting the drivers re-signed. Note that not all original drivers are WHQL signed and work just fine. Base slipstreams a tool as a workaround so unsigned drivers do not halt unattended setup.

Editing drivers, is fun... 
Working on a pack, is more fun.

When one looks at a driver, you will find that:

The driver.INF has info on its version, the supported HWIDs, and a section in there [sourcediskfiles] lists the files used to make a driver work.
Compare content of the unpacked driver against all "Copyfile" lines.
When you look at these entries, you know which files are essential.
You can then look whether the required files are present.
The NON essential files can be left out to save space, if you wish.

The driver.SYS file also has version info in it, and the internal name can be looked at too.
This is just a right-click away. One sometimes finds that it has a different internal name.
A team member reported and fixed a filename error in an original driver because he had looked at that.
The internal name of the SYS file was used in the INF, but the real name of the file was "wrong".

Also look at its size.
When you compare a driver against a similar driver (pretty often they use same name), you will want to look at the INF.
Besides HWIDs, version, the files used, one has to compare the rest of the txt in the INF files as well.
You see, if you want to MERGE a driver, then the registry entries have to match as well.
(MERGING means adding the HWIDs of one driver into another driver's INF, so as to not have to use two sets of driver files.)
If these registry entries are different while using same GUI Class keys, you can be almost sure merging won't work.
(You can easily spot the likeness and differences.)
If its SYS file has same name but is different in size, you can be almost sure merging won't work.

Now, if that driver is NOT an update of the one you compare it with, and you want to add it, then you will have to make a new section entry in mass storage INI.
One of the first considerations is the name it will be given for txtmode.
Look for existing ms_1_tag names and do not use an existing name if you add a driver.
You will have to look at supported HWIDs and compare these with the rest of the pack.

When you find you are able to merge, you edit the driver's INF, and add the HWIDs in the corresponding section in mass storage INI.

Sometimes, there are Updated Drivers which drop support for older hardware.
When you read its INF and find that new DLL files, extra SYS and extra INF files are required and used (like nodrv.inf), then one can edit out the duplicate HWIDs in the older driver, so that only the NEW one will cover these.
You will do that in Mass storage INI to avoid conflicts.
(Otherwise, during Txtmode stage of setup, two sys files would get loaded for the same Chip.)
You can also edit the older driver's INF (Sometimes this is a good solution).
This is done to avoid a clash during PNP stage which could occur when both are unsigned.
Of course, you can disable the older one altogether (Do Not Delete. You may have to re-enable at a later date.), but the newer one will sometimes not support some older chip versions and we've seen that old hardware has a tendency to outlive vendor support.

To Illustrate this, let's recount a real story.
It happened in Mass storage forum.
An update for LSI was requested, and Jaak took the task.
What could a look at files and the INF tell us?
Old LSI in [L-4] did not use pseudo device, and a 'tactic' like 'nodrv' or 'pseud' is noticed in relatively new drivers.
Then Jaak looked for matching HWIDs to avoid possible conflicts. At first, Jaak forgot to look through ALL drivers and therefore he did not notice there was a DELL using HWIDs this LSI listed.
So, he had looked for double HWIDs only in LSI, found them, but saw he should not edit [L-5] because the last line there told him it was for server.
Because of that oversight, there was an unnoticed DELL driver using HWIDs this updated LSI also listed, and this was pointed out by LSe.

LSe wrote:
In PnP driver install phase (about T=38 to T=34 in WXP installation) there is a ranking for each hardware ID to find the best matching driver. Presence/absence of signature (The CAT), version, matching ID (full/generic) and others make for each matching string from a *.inf file a rank value. The driver with the lowest resulting rank value will be installed.
You can see what happened in this phase by studying setupapi.log file in %windir% after the installation.
So for the PnP the presence of more then one matching string is no problem, normally the best will "win".
The problem is the txtmode setup phase. At this stage there is no ranking, only one driver with the same name can be loaded.
So if we want to keep the Dell and the native LSI version separated we must use the older generic version 1.20.18.00 in txtmode. Obviously it also supports the Dell controllers. If there is a Dell controller installed, this older unsigned driver will be replaced in the PnP phase by symmpi.sys 1.21.08.00. If "only" a LSI controller is installed it will be kept in PnP.
If we would use the Dell driver in txtmode an there is only a LSI controller installed, the PnP must downgrade it. I fear this wont work, but I will try out this cenario in the next days.

Jaak replied;
Trying to make txtmode match what pnp does later on, is what I thought what needs to be done, and it is not easy.
TXTmode uses the mass storage INI, and PNP uses the INF.
Some drivers are unsigned because the version of that sys supports several oems using same sys for their driver, and they got merged, which broke signing.
After somebody had pointed it out, I realised that when a conflicting signed one with matching generic HWID in INF is left signed (by not removing generics) then one can get BSOD.

Because of this finding, we had to make a choice..
In our example, the OEM Dell driver [D-3] has a valid signature unlike the native LSI [L-4].
The same Dell HWIDs were suppored in LSI [L-4].
The Dell driver has also a slightly higher version number (1.21.08.00) and a newer date than the original LSI files.
So? What influenced our choice?
If D-3 is signed, keep it.
If the SYS file size for the one in LSI is different, definatetly keep Dell.
Still more considerations were made.
Is there a choice between unsigned generic versus signed proprietary? Keep signed proprietary.
A choice between Subsys and 'proprietary" subsys? Keep signed 'proprietary" subsys.
The filter (mass INI) must be changed to not use non-signed LSI in [L-4] for hwid signed in [D-3]
One could say that generic HWIDs in signed driver should be kept, and unsigned generics can be tossed away if one has it covered by a signed driver. Then, Dell won. BUT... It actually LOST.
In some cases, one can make an edit to a signed driver's INF, and kill the generic in there, keeping the specified subs. You can do that if this one is proprietary, and the other supports a lot more generic and SUB_sys HWIDs as well.
The LSI driver was not signed, older, and therefore Jaak edited the Mass storage INI for [L-4] so that it did not list DELL's HWIDs.
One "could" go as far as editing its INF as well, but since Dell was newer, it was the DELL's INF that got edited to not support the generic. (Jaak did that because Dell, just like Compaq and other OEMs do, often use proprietary chips. And, the SYS they use, is most likely specific for that proprietary chip. Well, the size was different as well.)
LSI's can most probably handle the generics better than Dell's prorietary sys-file.
Will this be the best solution?
Time will tell.
====================================

TIPS.
If you want to modify a driverpack, extract it in a separate folder, because later on you have to repack it along with its INI.

You will have to look at the INI, because if you remove folders what got referenced in the INI, finisher would throw an error. (It will show in the LOG.)

BEWARE, you have to repack.
DPs_base uses the packs, not the extracted folders.
The name you give the repacked pack must follow the naming convention for a driverpack.
If you unpacked xxx7041, you could call it xxx7042 so as not to lose the original.
I figure you would want the original in case your modification was not succesful, and if it is, its higher version number will be useful.

When you repack; Don't merge more than one pack into a single archive.
If you do that, it will NOT work.

--------------------------------------------------------
Making a pack perfect is not easy.

A testing version was put on rapidshare by Jaak.
BUT; Jaak had made an error in mass storage INI when he added a driver for servers in [c]
In [C] section, for a driver intended only for SERVER, Ms_1_exc_disableIfOS= "wxp" was used.
This had caused a long list of disabled drivers.

JSe wrote
The problem starts after [AU-3], the first commented out is [C] CPQ32FS2.
Compared to the original 704 there some changes at this point 704 also had
[AU-4] and [AU-5] but no [C].
and have found the problem:
it's the option
ms_1_exc_disableIfOS= "wxp" under the [C] header. From this point on _all_
drivers that follow will be disabled, this seems to be a bug of the Base program. I have replaced this option by
ms_1_exc_skipIfOS = "wxp"

Meanwhile, Jaak had reported what he thought was a bug at the bugtracker..

Because of this, OverFlow wrote:
Because you don't understand something does not mean there is a bug in the
base.
disable was not what you wanted
ms_1_exc_skipIfOS= or set ms_count = 0 are to "Disable" a driver (from
your perspective)
the ms_1_exc_disableIfOS= "wxp" means something totaly different to the Base
program.
"load the driver for plug and play but; in a DISABLEd state during text mode
setup."
We have never needed to do this for any OS except windows 2000.
...

JSe (thinking that was meant for him) then wrote:
Yes, you are right, I did not completly understand this functionality, but please note:
1. It was not me, who used this ms_1_exc_disabledIfOS="wxp", I was only the one who wrote that this option caused the problem of commenting out _all_ drivers, not only that one, that had this option applied that follow this driver.
2. What I still would call a (possible) bug is that an option, regardless if it is usefull or not at the point it had been used acts on all the following drivers. And if you dont like the word bug then let's call this unpredictable behavior we should be aware of if it happens again.
Then Jeff wrote:
I disagree. It does what it was designed to do. It is neither a bug nor unpredictable.
It is used for w2k to prevent drivers from loading into ram to prevent ram from being used up and it does.
We have yet to add enough drivers to find these limits on XP or 2003 and is not needed or supported.
If you put Garbage Into the .inf file you will get Garbage Out (GIGO).
That is not a flaw with the base it's user error. The "meat" virus.
Let's agree it was just not well documented.

And then Jaak wrote
I was unaware of that too, and I made that error in [C]
One could call it "growing pains", and I really think this should be made "more robust" against such "user" error.
I would NOT have posted RUUDBOEK's text had this project not become open-source.
The end user has had the possibility to edit the INI for a while now, and little in depth information was public.
Give me the choice between PM or chat for help... Then I will choose public discussions. Always.

Long time ago, on other boards, some long discussions got summarised, and posted.
A summary like that was welcomed.
It came with all relevant links, and all posts or helpful parts of posts that got included were credited to the original poster.
You can bet there are people using snips of posted tips to write their own reference txt.
I am one of those...
---------
So, this is how this txt came to fruition.
We've now collaborated on this, and we hope it is informative.

Some quoted text was edited for clarity or grammar.
We're an international bunch, and English is only a second language to many.

We hope this helped you.
=======================

revision six, date 2007/05/16