ISYE 6414 - UNIT 5 QUESTIONS AND 100% CORRECT ANSWER 2024

ISYE 6414 - UNIT 5 QUESTIONS AND
100% CORRECT ANSWER 2024
WhatUareUthreeUproblemsUthatUvariableUselectionUtriesUtoUminimize?U-
UANSWERUhighUdimensionality,Umulticollinearity,UpredictionUvsUexplanatory




highUdimensionalityU-
UANSWERUInUlinearUregression,UwhenUtheUnumberUofUpredictingUvariablesUPUisUlarge,UweUmightUgetUbetterU

predictionsUbyUomittingUsomeUofUtheUpredictingUvariables.



ModelsUwithUmanyUpredictorsUhave...U-UANSWERUlowUbias,UhighUvariance



ModelsUwithUfewUpredictorsUhave...U-UANSWERUhighUbiasUbutUlowUvariance



predictionUriskU-UANSWERUaUmeasureUofUtheUbias-varianceUtradeoff



HowUdoUweUestimateUpredictionUrisk?U-UANSWERUweUcanUuseUanUapproachUcalledUTrainingURisk



trainingUriskU-
UANSWERUcomputeUtheUpredictionUriskUforUtheUobservedUdataUandUtakeUtheUsumUofUsquaredUdifferencesU

betweenUfittedUvaluesUforUsubUmodelUSUandUtheUobservedUvalues



IsUtrainingUriskUbiased?UwhyUorUwhyUnot?U-
UANSWERUYes,UtheUtrainingUriskUisUaUbiasedUestimateUofUpredictionUriskUbecauseUweUuseUtheUdataUtwice.UO

nceUforUfittingUtheUmodelUSUandUonceUforUestimatingUtheUpredictionUrisk.UThus,UtrainingUriskUisUbiasedUupw
ard.



TheUlargerUtheUnumberUofUvariablesUisUforUtrainingUrisk....U-UANSWERUtheUlargerUtheUtrainingUrisk



WhatUcanUweUdoUsinceUtheUtrainingUriskUisUbiased?U-
UANSWERUWeUneedUtoUcorrectUforUthisUbiasUbyUpenalizingUtheUtrainingUriskUbyUaddingUaUcomplexityUpenalt

y.

, 4UtypesUofUVariableUSelectionUCriteriaU-UANSWERUMallow'sUCp,UAIC,UBIC,ULOOCV



Mallow'sUCpU-
UANSWERUThisUisUtheUoldestUapproachUtoUvariableUselection.UThisUassumesUthatUweUcanUestimateUtheUvari

anceUfromUtheUfullUmodel,UhoweverUthisUisUNOTUtheUcaseUwhenUpUisUlargerUthanUn.



AkaikeUInformationUCriterionU(AIC)U-
UANSWERUAUmoreUgeneralUapproach,UforUlinearUregressionUunderUnormalityUthisUbecomesUtrainingUriskU+U

penaltyUthatUlooksUlikeUMallow'sUEXCEPTUtheUvarianceUisUtheUtrueUvarianceUnotUtheUestimate.



Leave-One-OutUCrossUValidationU(LOOCV)U-
UANSWERUThisUisUaUdirectUmeasureUofUpredictiveUpower.UThisUisUjustUlikeUMallow's,UexceptUtheUvarianceUisU

forUtheUSUsubmodel,UnotUtheUfullUmodel.UTheULOOCVUpenalizesUcomplexityUlessUthanUMallow'sUCp.



ToUcorrectUforUcomplexityUforUGLM,UwhatUcanUweUuse?U-UANSWERUAICUandUBIC



AICUvsUBICU-UANSWERUBICUisUsimilarUtoUAICUexceptUthatUtheUcomplexityUisUpenalizedUbyUlog(n)/2



AnUimportantUaspectUinUpredictionUis....U-UANSWERUhowUitUperformsUinUnewUsettings.



We'dUlikeUtoUhaveUpredictionUwith...U-UANSWERUlowUuncertaintyUforUnewUsettings.



IfUp(predictors)UisUlarge,UisUitUfeasibleUtoUfitUaUlargeUnumberUofUsubmodels?U-UANSWERUNo



IfUpUisUlarge,UwhatUcanUweUdoUinstead?U-
UANSWERUWeUcanUperformUaUheuristicUsearch,UlikeUstepwiseUregression




IfUpUisUsmall,....U-UANSWERUfitUallUsubmodels



ForwardUstepwiseUregressionU-
UANSWERUweUstartUwithUnoUpredictorUorUwithUaUminimumUmodel,UandUaddUoneUpredictorUatUaUtime