See this tool built by jalasama on discord: https://sufftea.github.io/manifold_calibration/

@Bayesian this looks cool but how does it work (what assumptions does it make - average value of shares / final value of shares in one bet / bet sizing / etc)?

I think the key issue is that a Brier score is not super meaningful with a TRADING platform. Imagine you are a PERFECT trader, and 100% of your YES bets resolve YES and 100% of your NO bets resolve NO. You will then have a horrible Brier score! (imagine a plot with the green dots all in a line on the top and the red dots all in a line on the bottom of the graph).

@Bayesian Thank you for sharing! This is very cool.

@benshindel brier score is super non-meaningful for prediction markets, but people find calibration interesting (it really doesn’t imply much about predictive skill, but it’s still fun to see).

This site is simple and treats all bets equally based on the market price post bet. Only for binary atm IIUC

okay wtf I keep getting notifs about comments here but they're invisible. not even hidden, just completely MIA. anybody else?

presumably you mean the comments by me and plainboomerang? Hmm i see them down in the comments, possible bug

@Bayesian Yeah, I see the comments too

https://manifold.markets/calibration feels not fair that manifold has calibration, but users do not.

bring it back or i lose 1500 mana : (

lol

@BayesianTom I need it the back here PRONTO on the double!!! It is so important for me to see it, I am adding it for the LinkedIn profile and the future employing needs the confirmation of good decisions I am making!

We need all the whales to assemble and save us. We can meme it back!

#SAVECALIBRATION

@Bayesian You are a NO holder. I need you to exit the dark side.

@PlainBG "Vote (or write comments) on values, bet on beliefs"! I hope calibration comes back but I don't expect it to.

@Bayesian Unless if the developers consult the probability to decide if it should be prioritized...

@PlainBG if the developpers of a prediction market don’t understand that the market probability doesn’t represent the degree of support for a policy we‘re ngmi

@Bayesian It's at minimum a measure of community expectation. Low expectation -> no results

@PlainBG That would be a big L moment. I don’t expect it

This is super bad that Manifold removed calibration.

I thought I was building a track record that is EASILY UNDERSTANDABLE BY OUTSIDERS! I could even put on a resume!

Yes, profit is good, but it's affected by other stuff, like how much did you put on Manifold in the first place.

I want Calibration back! I'm going to bet NO on this as a form of bounty on it's return

Blasphemy!

Does the calibration page actually work for you guys? It doesn't for me, on my own profile anyway.

And here I was hoping they'd make an expanded version of the calibration and predictive skill stats with interactive graphs of changes over time and comparisons to other users that would be prominently displayed on your profile as a badge of trustworthiness 😞

I had an A calibration! NEED IT BACK NOW