Decision Making based on the Monte Carlo Method

Decision Making Based on the Monte Carlo Method (and Software Instruction Guide)

Part I: Design Stage

1. 
   Define the system objectives and list any requirements.
   
   1. 
      The system objectives state the purpose of the design.
      
   2. 
      Requirements include any task the system or design must perform as well as any constraints such as a budget or schedule.
      
2. 
   Define an option space or possible design configurations.
   
   1. 
      Select a few options to analyze. Since an option can include an infinite number of choices, it is feasible to analyze the most likely choices. (Note: However, this process may lead to a sub-optimal choice.)
      
   2. 
      Define the parameters of each option i.e. the numerical quantities that specify design elements.
      

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

• 
  Define the system objectives and list any requirements: Suppose you need to select the I-beam that best fulfills the following requirements:
  
  • 
    Member is constructed from aluminum
    
  • 
    I-beam is simply supported at both ends
    
  • 
    I-beam is uniformly loaded with a distributed force equal to 3775 N/m
    
  • 
    I-beam cannot deflect any more 0.60mm
    

The setup:

• 
  Parameters:
  

Cross Section of I-Beam



Part II: Evaluation Measures (EMs)

1. 
   Based on the design objectives and requirements, create a list of evaluation measures for your system.
   
   1. 
      An evaluation measure is a numerical quantity that allows one to grade some aspect of the design.
      
   2. 
      It is important that the evaluation measures are independent, meaning that two measures do not evaluate the same quality or a combination of the same qualities.
      
2. 
   Define the details of each evaluation measure.
   
   1. 
      Is the EM deterministic or probabilistic?
      
      1. 
         A deterministic EM is one whose values are fully expressed as one number. (For Example: The safety rating of a car is 4 stars out of 5.)
         
      2. 
         A probabilistic EM is one whose values are expressed as a range. (For Example: The length of a beam can be 2 meters +/- 10 cm.)
         
   2. 
      Result:
      
      1. 
         If the EM is deterministic, determine if higher option values are more favorable.
         
      2. 
         If the EM is probabilistic, also determine if higher option values are more favorable. Next, determine the risk inclination.
         
         1. 
            There are 3 risk states, risk averse, risk neutral and risk seeking. After selecting a risk inclination determine the extent of the risk inclination, rho or “ρ.” If one is risk averse then ρ is positive and if one is risk seeking then ρ is negative, and if one is risk neutral then ρ= infinity. As a general scale for ρ, a value of 5 is moderate and a value of 1 is extreme. This value must then be de-normalized (multiplied by the range of the EM: High-Low). For further details reference the graphs below which display the affect of one’s risk inclination (rho) on the utility value:
            

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

List Evaluation Measures: The evaluation measures for this decision will include

• 
  Cost
  
• 
  Deflection
  

Evaluation Measure Description:

• 
  Cost is a deterministic EM since the price per beam will be used in the analysis. For cost a lower option value is more preferable.
  
• 
  Deflection is a probabilistic evaluation measure. Since there are tolerances in the beam’s dimensions, the performance of each beam needs to be simulated. For deflection a lower option value is more preferable.
  
  • 
    For deflection this example will be risk averse with a rho=5.
    

________________________________________________________________________
CHECKLIST: Before proceeding be sure to have the following information:

• 
  Options with parameters
  

• 
  Evaluation Measures and details including:
  

• 
  Deterministic / Probabilistic
  
• 
  Value preference (are higher or lower option values more preferable?)
  
• 
  Risk Inclination
  

________________________________________________________________________
Part III: Evaluation Measure Details

1. 
   If an EM is probabilistic then that particular factor of the design needs to be modeled as a random variable. To complete this step a simulation must be performed to determine the distribution. To find the distribution of the EM:
   
   1. 
      Open MATLAB and change the working directory to where EQUATION.M and LOOPY.M are saved.
      
   2. 
      Open “EQUATION.M” and using this file define the variables that make up the equation which determines your EM. Then set the result vector, RV, equal to that equation. Be sure to save the file under a new, descriptive title. (YOURTITLE.M) Note: MATLAB is case-sensitive. It is important to note the order of the variables because the same order will be used in the simulation. See comments in EQUATION.M.
      

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

The equation to determine the deflection of the beam is (Youngs = Young’s Modulus and MomentI = moment of inertia)

Due to the complexity of this equation it will be broken down into two equation files. One will determine a distribution for MomentI since the moment of inertia is a value determined by multiple dimensions from the beam’s cross section. The second equation will compute the rest of the deflection equation as a distribution. Then both distributions will be combined which yields the deflection distribution.
Equation File One: Moment of Inertia

Equation File Two: Loading

________________________________________________________________________

1. 
   Run “LOOPY.” Loopy is the MATLAB program that models the variables defined in the Equation file as distributions using the Monte Carlo Method. You will be asked:
   

1. 
   “How many Variables?” The answer is equal to the number of variables previously defined in “YOURTITLE.M.” (In Loading.m, number of variables = 2)
   
2. 
   “How many Simulations?” 10,000 to 100,000 Monte Carlo simulations will return satisfactory results.
   
3. 
   “How many bins?” After each variable is simulated a histogram of that variable appears so that you can check that the variable was correctly modeled after the desired distribution. The “bins” refers to the number of bars plotted in the histogram. 50 bins will usually produce satisfactory results.
   

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

Below is a list of the prompts from “LOOPY”

________________________________________________________________________

Next, a menu bar of the available distribution types will appear. Select the desired distribution. Then “LOOPY” will prompt you to input information required to define that distribution. Enter this information and then press ENTER. A histogram of the random variable will appear. View the graph then press ENTER to continue modeling the remaining variables. After all simulations have been run, “LOOPY” will prompt you to type the name of “YOURTITLE.M.” In single quotes type the name of the equation file saved (without the ‘.M’) and then press ENTER. A final histogram of the result will be displayed for the option. Set the Option name to equal RV (Option1=RV) press ENTER.

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

For deflection you will need to run the loopy program twice for each option. For this example, the normal distribution will be used to model the variables. A normal distribution will ask you to input the mean and the standard deviation of your variable. The nominal value will be input as the mean and the +/- tolerance divided by two will be the value for the standard deviation. Thus, if you are simulating the length of the beam the mean value equals 3m and the standard deviation equals 0.02m or (0.04 / 2). (See values from page 2 for further details)

• 
  The first loopy run will be used to compute the moment of inertia for the beam. Follow the instructions and input the appropriate information. Next type “Moment1=RV;” and this will set the distribution of the moment of inertia equal to the title Moment1 or the moment of inertia for option 1.
  
• 
  Next run loopy again except use it to compute the load portion. After loopy is complete set “Load1=RV;” and this will set the load portion of the deflection equal to the title Load1 or Load of option 1.
  
• 
  Finally, type “Deflection1=Load1./Moment1;” and this step will complete the deflection equation thus returning a distribution for the deflection of the option. (See README_MATLAB for an explanation of the “.” operator.)
  
• 
  Type “hist(Deflection1,50);” and a histogram of the distribution will be plotted with 50 bins and it should look similar to:
  

________________________________________________________________________

4. 
   Perform the “NORMFIT” function on the simulated option. Using this syntax: [mu,sigma,muci,sigmaci] = normfit(RV,alpha), choose a confidence level (95% conf= 0.05=alpha); this function will return four items:
   
   1. 
      mu: the mean of your sample distribution
      
   2. 
      sigma: the standard deviation of your sample distribution
      
   3. 
      muci: the max and min the mu is restricted to, dependant on your confidence level.
      
   4. 
      sigmaci: the max and min the sigma is restricted to, dependant on your confidence level.
      

For further details simply type “help normfit” at the MATLAB prompt.

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

Normfit Results for Option 1

Repeat this process again to obtain the deflection distribution and normfit information for the other options.

________________________________________________________________________

5. 
   Using the information from “NORMFIT” and your risk inclination chose a Certainty Equivalent for your option. The certainty equivalent is a pseudo expected value for a distribution that has been adjusted based on one’s risk inclination. For example, if the given distribution is for deflection and the mean is 1.0mm and less deflection is preferable, then the certainty equivalent for someone who is risk averse could be 1.3mm. Likewise, for the same distribution, if the person is risk seeking then his certainty equivalent may be 0.8mm. One way to determine the certainty equivalent is to use the muci values. If higher values are more preferable for the EM and you are risk averse then you might want to choose the lower of the two muci values to be on the safe side.
   

6. 
   Select a HIGH and a LOW limit for the evaluation measure. HIGH is the largest acceptable value for the EM and likewise LOW is the smallest acceptable value. This range must encompass all of your option scores. Any option that does not fall between the HIGH and LOW values may be discarded.
   

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

Continuing with Option 1 …

• 
  The Certainty Equivalent for deflection will be taken from the muci information. Since the example is risk averse the higher of the two muci values will be used as the certainty equivalent. Certainty Equivalent = 0.0005259m (Continue on and determine a CE for each option).
  
• 
  Select a HIGH and a LOW value for the evaluation measure
  
  • 
    HIGH= 0.0006m
    
  • 
    LOW= 0.00045m
    

In addition it should be noted that since Deflection is probabilistic, the software will allow you to use the standard deviation of the deflection as an evaluation measure as. In this example the standard deviation of deflection will be used as an evaluation measure since it is important for the distribution to be tightly packed around the mean.

________________________________________________________________________

2. 
   If an EM is deterministic:
   
   1. 
      Select a HIGH and a LOW limit for the evaluation measure. HIGH is the largest acceptable value for the EM and likewise LOW is the smallest acceptable value.
      

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

For the cost evaluation measure:

• 
  Select a HIGH and a LOW value for the evaluation measure
  
  • 
    HIGH= $130.00
    
  • 
    LOW= $120.00
    

­­­­­­­­­­­________________________________________________________________________
CHECKLIST: Before proceeding be sure to have the following information:

• 
  For deterministic know the Evaluation Measure’s:
  
  • 
    HIGH
    
  • 
    LOW
    

• 
  For probabilistic know the Evaluation Measure’s:
  

• 
  HIGH
  
• 
  LOW
  
• 
  Certainty Equivalent
  

________________________________________________________________________
Part IV: Determining Option Grades

1. 
   To determine the option grades, a MACRO for MS Excel will be used. Before proceeding, read the README_XL file for disclaimers.
   
   1. 
      Begin by double-clicking on the Excel file “Decision Making Macro.”
      
   2. 
      Next, go to the top tool bar and select: “Tools”  “Macro”  “Run Macro.”
      
   3. 
      Select “CreateEMs” and click RUN.
      

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

4. 
   Follow the instruction boxes and input the necessary information. (Have the information from the previous “Checklists” on hand to expedite this process)
   
5. 
   Once complete, SAVE AS the file with a new name descriptive of your project.
   
6. 
   Go through the newly created EM sheets and fill in the score boxes shaded in blue. All boxes that are shaded in blue must be filled in for the later macros to work properly. (Any boxes shaded in green are optional and contain only reference information to benefit the user)You should note that the Macro allows you to use sigma (standard deviation) as an evaluation measure if the EM is probabilistic. Refer to Appendix I for further details. As you input values, the grade of the options should appear in the appropriate boxes.
   
   1. 
      The Grade of an option is a number that takes into account the option score, your risk inclination, and the range of the option.
      
   2. 
      For information on the exact formula used to compute the option grade, please reference appendix II.
      

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

7. 
   SAVE again.
   
8. 
   Once all the evaluation measure information has been entered, select the “Final” sheet.
   
9. 
   Inputting the weights of the evaluation measures into the blue boxes: You can choose to enter the weights yourself (arbitrarily) or you can use the weight macro to guide you along.
   
   1. 
      Before proceeding, for any EM that you do not wish to include, put 0’s (zeros) in their weight columns.
      
   2. 
      To run the weighting macro: “Tools”  “Macro”  “Run Macro.”
      
   3. 
      Select “WeightCalc.”
      
   4. 
      The Weight of an EM is a measure of how important that EM is with regard to the other EMs.
      
   5. 
      The value of a weight is always between 0 and 1.
      
   6. 
      The sum of the weights of all the EMs is equal to 1.
      
   7. 
      Below are the types of weight elicitation techniques available in the macro:
      

1. 
   Direct Methods
   
   1. 
      Each method follows these steps:
      
      1. 
         Numerically assign a rank order to the evaluation measures. List the EMs from most important with a value of 1 to the least important. (This first step should always be performed regardless of the weight elicitation technique that you choose to employ)
         
      2. 
         Use one of the following formulas for assigning weights
         

Rank Sum: r_i is the rank of the ith EM, K is the total number of EMs

Rank Exponent: r_i is the rank of the ith EM; K is the total number of EMs; z is an undefined measure of the dispersion in the weights



Rank Reciprocal: r_i is the rank of the ith EM, K is the total number of EMs


Rank Order Centroid: r_i is rank of the ith EM, K is the total number of EMs

2. 
   Indirect Methods
   
   1. 
      Trade Offs: (not available in the macro)
      
      1. 
         Rank EMs in order of their overall value
         
      2. 
         You are asked if overall weight of the second EM is as great as the weight from the lowest to some intermediate point of the value scale of the first EM.
         
      3. 
         The third EM can now be compared to intermediate points on either the first or second ranked EMs and so on…
         
      4. 
         Method works well when the value curves are firmly established and when the value curves are continuous
         
   2. 
      Balance Beam Approach:
      
      1. 
         Establish a rank order of the overall weights of the evaluation measures.
         
      2. 
         A series of questions is posed beginning with “Is the overall weight of the first EM (a) greater, (b) less than, or (c) equal to the overall weight of the second and third EMs
         
      3. 
         If answer is “less than” then the third EM is dropped and replaced by the fourth EM. (If “greater than” then the fourth EM is added to the second and third )
         
      4. 
         The goal is to establish a series of equations that define the weights for all of the evaluation measures.
         
      5. 
         The least valued EM is given a weight of 1 and then you are asked to assign a weight to the second least weighted EM then this information is used to solve the system of equations
         
      6. 
         The results are then normalized into weights with values between 0 and 1
         

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

The rank order of the evaluation measures is as follows:

1. 
   Deflection
   
2. 
   Cost
   
3. 
   Deflection Deviation
   

Remember the numbers associated with the EMs because the macro will prompt you for these values.

Below is a table which includes the weights determined by all of the direct weight methods. While you only need to select one weight method the table is provided for your edification.

Method	Rank Sum	Rank Exponent*	Rank Order	Rank Reciprocal
Cost	0.3333333	0.341081377	0.2777778	0.272727273
Deflection	0.5	0.417737668	0.6111111	0.545454545
Sigma of Deflection	0.1666667	0.241180955	0.1111111	0.181818182

*For Rank Exponent a dispersion value of 0.5 was used.

And by the Indirect Method of Balance Beam:

After completing the instructions of the macro for the balance beam method the following box will appear:

This message box contains the system of equations that you will use to determine the weights. Since the lowest EM is set to one go to the next equation and ask yourself how much greater Cost is than the Sigma of Deflection. For this example, Cost will equal 2. Use these values to complete the final equation which gives Deflection a value of 3. Finally, normalize these values into weights between 0 and 1. In the end:

• 
  Deflection weight = 0.5
  
• 
  Cost weight = 0.33
  
• 
  Sigma of Deflection weight = 0.17
  

________________________________________________________________________

10. 
    Go to the top tool bar and select: “Tools”  “Macro”  “Run Macro”
    
11. 
    Select “MakeFinal” and click RUN. This will execute the Macro that computes the final grades for the options.
    
    1. 
       The final grade is computed using the following equation:
       

This equation is summed over each evaluation measure.

12. 
    Save
    

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

Using the weights from Rank Exponent, the final calculated grades are:

Evaluation Measure	Weight	IBeam1	Weighted Grade	IBeam2	Weighted Grade	IBeam3	Weighted Grade
Cost	0.3411	0.6	0.204648826	0.66	0.225113709	0.7	0.238756964
Deflection	0.4177	0.519	0.216799826	0.3535	0.147676309	0.2845	0.118842461
Sigma of Deflection	0.2412	0.7207	0.173820469	0.6238	0.150453745	0.5746	0.13859396
Final Grade			0.595269121		0.523243763		0.496193385

________________________________________________________________________

Part V: Sensitivity

1. 
   Now that the final grades have been determined it is important to perform a sensitivity analysis.
   
   1. 
      A sensitivity analysis will allow you to see if you have selected a good weight value for an evaluation measure.
      
   2. 
      The analysis works by varying the weight of one EM from 0 to 1 while adjusting the weights of the other EMs to keep their proportions the same. The final grades of each option are re-computed then graphed.
      
   3. 
      To run the sensitivity analysis go to “Tools”  “Macro”  “Run Macro”
      
   4. 
      Select “Sensitivity” and click RUN.
      
   5. 
      For each EM a new sheet containing the sensitivity plot will be created.
      
2. 
   Plot Review
   
   1. 
      The “Sensitivity Analysis” is primarily concerned with analyzing the area around the chosen weight for the evaluation measure. (If the weight of a particular evaluation measure is 0.5 then the x axis of the plot should go from 0.4 to 0.6.) Therefore, readjust both the “x” and the “y” axis for each new plot. To do so place the cursor over the axis you desire to alter and right click. A menu bar should appear and select “FORMAT AXIS”. An option box will appear and you need to select the tab titled “SCALE”. On the scale page input the minimum and maximum values for your graph that should be displayed and click “OK.”
      
   2. 
      For each new plot look for any intersections near the chosen weight of the varied evaluation measure.
      
   3. 
      If the plotted lines of the options intersect or are close together at the point of interest you may wish to re-evaluate your weights for a clearer decision.
      
   4. 
      SAVE
      

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

Below is the sensitivity analysis plot for the Cost evaluation measure:

Note that the x and y axes on the chart were altered to display the appropriate range of values.

________________________________________________________________________

3. 
   Review the final grades again and take into consideration the sensitivity analysis and decide on the best option for the design.
   

Example Example Example Example Example Example Example Example Example Example Example Example Example Example Example

After reviewing the final grades and the sensitivity analysis it is clear that I-Beam 1 is the best option for the design.

Appendix I
Standard Deviation Notations:

1. 
   For all continuous EMs, you have the option of including sigma (standard deviation) of the distribution as an evaluation measure.
   
   1. 
      While the Macro automatically makes a space for sigma, you can eliminate its effect by giving it a weight of zero on the “Final” page.
      
   2. 
      If you are unsure whether or not to include the standard deviation the following information may be helpful:
      
      1. 
         First, it is important to realize that all designs are unique. Therefore, using sigma as an evaluation measure is situational to the specific design. (Sigma should be considered as an EM if the tolerances around the mean need to adhere to a certain restriction).
         
      2. 
         The standard deviation may not be useful as an EM if all points of the distribution meet your set criteria… In general there are three criteria types.
         
         1. 
            All points > X
            
         2. 
            All points < X
            
         3. 
            X
            1. 
               If not all points on the distribution meet your criteria, it may be more useful to consider the number of points that fail to meet the criteria as an evaluation measure instead of the standard deviation. To determine the number of points that fail to meet your set criteria use the “HISTC” function in MATLAB. (see README_MATLAB for details)
               

Appendix II

1. 
   Grade Determining Equations:
   

1. 
   Deterministic or Risk Neutral evaluation measure
   
   1. 
      Higher option scores preferable:
      

2. 
   Lower option scores preferable:
   

2. 
   Probabilistic evaluation measure:
   
   1. 
      Higher option scores preferable:
      

2. 
   Lower option scores preferable:
   

Page of