No it’s not. The scientific consensus is that machines don’t have values and never will, and controlling them is an engineering problem. Unfortunately there’s a bunch of whackos at some frontier labs who think they’re the scientific community.
I disagree. Ya, simply adding tactile sensing to a trained policy without retraining won’t help, but that’s a training limitation. You can condition the policy on tactile observations while keeping joint position as the action space. The policy observes contact forces (or raw tactile) and adjusts its position targets.
Alternatively, the policy could also predict desired contact forces directly and use a low level force controller.
Deformable tips implicitly encode force through deformation, but the position-to-force mapping isn’t unique. Grasping a rigid cube vs. a soft sponge of the same size at the same gripper position produces different forces.
For the same geometry , friction and acceleration also rapidly change the required force to prevent slippage.
Ultimately, with position only policy your must learn these variations from more and more demonstrations.
Given a rough dynamics model (learned or MB) and successful trajectories and goal states, we can use backward rechabililty with the current policy to approximate a reachable set. Then can we construct a value function estimating the prob of reaching the goal states within some horizon H ( bellman recursion for small states or Monte Carlo rollouts)? If so we could use alpha level sets of this value function for different curriculum levels.
This assumes we are able to start the env at some specified state. The main issue I see is likely approximating these sets for large state spaces.
The issue with AI is the people, not the tech. Devs have devoted their lives to a craft. Instead of focusing on how new tools can help them achieve the previously impossible, we have huge swaths of people who have never accomplished anything frothing at the mouth at the prospect of obsoleting people better than them. Directly spurred on by the execs of top "labs" marketing to the lowest of men.
(1/7) I'd like to provide some additional context for the remarks that can be found in this interview.
(2/7) My discomfort with the attention being paid to existential risk---and with the ongoing dominance of the public discussion of such risks by a few so-called thought leaders---is that there's an elephant in the room that is being missed. That elephant is the effect of all of this discussion on young people, and indeed the effect---beginning with social media a decade ago---of information technology on young lives broadly speaking.
(3/7) I recently read an interview by Jill Lepore in which she reports on discussions that she had recently with a group of college students: "They had just three things to say. One, social media completely destroyed their childhoods. Two, AI is destroying their college education and their likely prospects for employment. And three, what the hell are we all doing?" To add a fourth bullet that I've heard in similar discussions: "I'm not going to have children, because the existential risk is just too high."
(4/7) Please contemplate such a statement, those of you who think that it's OK that the public, widely diffused, discussion coming from seemingly authoritative figures is so adamant about existential risk.
(5/7) Let me add that I have little interest in a debate about what "superintelligence" might mean, or whether we're currently seeing the emergence of superintelligence. For what it's worth, I would certainly agree that the solution of extremely challenging math problems (for example) exhibits a very impressive form of intelligence, although I would want to argue that such intelligence is impossible to dissociate from the collective of humans that provided data to the machine learning system (which itself has become a kind of collective).
(6/7) Such philosophical debates are entertaining and even important, but when they then head in the direction of "when such superintelligence emerges, there's a good chance that it will want to kill all humans", I say "science fiction". Again, I don't think that it's bad to consider such thoughts, but I strongly believe that it's bad for society, and especially young people, for such thoughts, unaccompanied by any serious real-world analysis, to become a dominant form of discourse of the era.
(7/7) We are damaging human lives, and the consequences of that damage will no doubt be a major theme of the next several decades.
I love when robots move like robots should instead of human teleoperated or programmed behaviour.
RL can show the real potential of a given embodiment, beyond human priors.
That’s not true. Capability should come with efficiency.
Have you seen a kid grasping all the objects in the world before he manipulates a new object?
Like compression in information theory, once you’re data-efficient, you’ve captured the structure that leads to generalization.
the issue isn’t “RL”, the issue is that you’re doing RL on poor proxies and then are surprised when the algorithm does what you asked (whether explicitly or implicitly). the problem is fundamentally one of task specification.
I’ve been seeing agentic robots (coding agents fine-tuning code) on my TL and in recent papers, so I tried it on a paper I’m working on.
The task was to use a belief space over the receiver’s grasp (left arm) before transferring an object. In one of the test cases, the left arm collides with the cuboid and moves it before the grasp.
I gave 5.6 Sol High (I’m broke) the paper and the full repo (everything is model based) and put it in the loop to fix the issue. Sol moved the hand behind the cube, which incorrectly solves the collision, and verifies the object was returned because of the weak grasp.
With some steering it is able to solve the problem. It would be a fun rich man's expirment to see how different models would reason through this.
Modern Americans, you are not responsible for slavery. You are not responsible for stealing the land and killing almost all native Americans. Those things happened long ago, before you were born. But you are responsible for the stealing of land and the genocide of Palestinians by the state of Israel. These are directly caused by your bombs and your votes today.