β€œν™•λ₯ κ³Ό 톡계(MATH230)” μˆ˜μ—…μ—μ„œ 배운 것과 κ³΅λΆ€ν•œ 것을 μ •λ¦¬ν•œ ν¬μŠ€νŠΈμž…λ‹ˆλ‹€. 전체 ν¬μŠ€νŠΈλŠ” Probability and Statisticsμ—μ„œ ν™•μΈν•˜μ‹€ 수 μžˆμŠ΅λ‹ˆλ‹€ 🎲

4 minute read

β€œν™•λ₯ κ³Ό 톡계(MATH230)” μˆ˜μ—…μ—μ„œ 배운 것과 κ³΅λΆ€ν•œ 것을 μ •λ¦¬ν•œ ν¬μŠ€νŠΈμž…λ‹ˆλ‹€. 전체 ν¬μŠ€νŠΈλŠ” Probability and Statisticsμ—μ„œ ν™•μΈν•˜μ‹€ 수 μžˆμŠ΅λ‹ˆλ‹€ 🎲

IntroductionPermalink

<MLE; Maximum Likelihood Estimation>은 μ§€κΈˆκΉŒμ§€μ˜ μ ‘κ·Όλ²•κ³ΌλŠ” 사뭇 λ‹€λ₯Έ μ ‘κ·Ό 방식이닀. <MLE>λŠ” statistical inferenceλ₯Ό μˆ˜ν–‰ν•˜λŠ” λ§Žμ€ 접근법 쀑 ν•˜λ‚˜λ‘œ, statistical approach에 μƒˆλ‘œμš΄ μ² ν•™κ³Ό μ‹œμ•Όλ₯Ό μ œκ³΅ν•œλ‹€ 😁


Example.

ν™•λ₯  pλ₯Ό μ •ν™•νžˆ μ•Œμ§€ λͺ»ν•˜λŠ” p-coin을 10번 λ˜μ§„λ‹€κ³  ν•˜μž. 10번의 κ²°κ³ΌλŠ” μ•„λž˜μ™€ κ°™λ‹€.

H H T H T T H H H T

μ•žμ—μ„œ 배운 <proportion estimation>의 λ°©λ²•μœΌλ‘œ μ ‘κ·Όν•˜λ©΄, pλŠ” Point Estimator p^=6/10으둜 μΆ”μ •λœλ‹€.


μ΄λ²ˆμ—λŠ” <MLE>의 λ°©μ‹μœΌλ‘œ μ ‘κ·Όν•΄λ³΄μž! λ¨Όμ € 10의 동전 λ˜μ§€κΈ°κ°€ μœ„μ™€ 같이 λ‚˜μ˜¬ ν™•λ₯ μ€ μ•„λž˜μ™€ κ°™λ‹€.

P(H,H,T,…,H,T)=p6q4

μœ„μ˜ 식을 λ‹€μ‹œ μ“°λ©΄, L(p)=p6(1βˆ’p)4둜, λ™μ „μ˜ ν™•λ₯ μ΄ p일 λ•Œ β€œH H … H Tβ€μ˜ κ²°κ³Όλ₯Ό 얻을 ν™•λ₯ μ„ μ˜λ―Έν•œλ‹€.

이제, 이 ν•¨μˆ˜ L(p)λ₯Ό maximize ν•˜λŠ” pλ₯Ό κ΅¬ν•΄λ³΄μž. 방법은 κ°„λ‹¨ν•˜λ‹€. κ·Έλƒ₯ p에 λŒ€ν•΄ 미뢄방정식을 ν’€λ©΄ λœλ‹€. μ΄λ•Œ, κ³„μ‚°μ˜ 편의λ₯Ό μœ„ν•΄ logλ₯Ό λ¨Όμ € μ·¨ν•΄μ£Όμž.

β„“(p)=log⁑(L(p))=6log⁑p+4log⁑(1βˆ’p) dβ„“(p)dp=6pβˆ’41βˆ’p=0β†’p=6/10

즉, p=6/10이 β€œH H … H Tβ€λΌλŠ” κ²°κ³Όκ°€ λ‚˜μ˜¬ ν™•λ₯ μ„ Maximizeν•˜λŠ” ν™•λ₯ μ΄λΌλŠ” 말이닀!

이제 <MLE>λ₯Ό μˆ˜ν•™μ μœΌλ‘œ 정리해 λ‹€μ‹œ μ‚΄νŽ΄λ³΄μž!


MLE; Maximum Likelihood EstimationPermalink

Theorem. MLE for Bernoulli case

Let X1,…,Xn be a Ber(p) Random Samples, with iid.

Then, the likelihood function L(p;x1,…,xn) would be

L(p;x1,…,xn)=f(x1;p)β‹―f(xn;p)=px1(1βˆ’p)1βˆ’x1β‹―pxn(1βˆ’p)1βˆ’xn=pβˆ‘xi(1βˆ’p)nβˆ’βˆ‘xi

Take log on it!

β„“(p)=βˆ‘xiβ‹…log⁑p+(nβˆ’βˆ‘xi)β‹…log⁑(1βˆ’p)

Take derivative for p!

dβ„“(p)dp=βˆ‘xipβˆ’nβˆ’βˆ‘xi1βˆ’p=0

when solve the equation, then

p=βˆ‘xin=xΒ―

Theorem. MLE for Normal case

Let X1,…,Xn be a N(ΞΌ,1) Random Samples, with iid.

Find the MLE of ΞΌ!

L(ΞΌ;x1,…,xn)=f(x1;ΞΌ)β‹―f(xn;ΞΌ)=(12Ο€)nexp⁑(βˆ’βˆ‘(xiβˆ’ΞΌ)2/2)

Take log on it!

β„“(ΞΌ;β‹―)=nβ‹…log⁑(12Ο€)βˆ’βˆ‘(xiβˆ’ΞΌ)22

Take derivative for ΞΌ!

dβ„“dΞΌ=βˆ‘(xiβˆ’ΞΌ)=0

when solve the equation, then

ΞΌ=xΒ―

이제 λ‹€μŒ ν¬μŠ€νŠΈλΆ€ν„° ν†΅κ³„ν•™μ˜ κ½ƒπŸŒΉμ΄λΌκ³  ν•  수 μžˆλŠ” <κ°€μ„€ κ²€μ •; Hypothesis Tests>에 λŒ€ν•΄ 닀룬닀!! 😁 μš°λ¦¬κ°€ μ§€κΈˆκΉŒμ§€ μˆ˜ν–‰ν•œ β€œμΆ”μ •(Estimation)”을 ν™œμš©ν•΄ μ˜μ‚¬κ²°μ •μ„ λ‚΄λ¦¬λŠ” 것이 λ°”λ‘œ <Hypothesis Test>λ‹€!

πŸ‘‰ Introduction to Hypothesis Tests