
I will "format" my question in a way to make it easier for you to understand

It will be structured as follow:
Goal
How I am trying to achieve it
Code
Possible Error Zones (My opinion)
Question
Goal:
I want to select multiple objects within a box or a rectangle. This box is resized whenever the user drags the mouse somewhere. Here is an example: https://dl.dropboxusercontent.com/u/949 ... .46.53.wmv (the objects are in 3D)
The picked NewtonBodies would be put into a vector containing all the picked objects (or NewtonBodies)
How I am trying to achieve it:
I found that NewtonWorldConvexCast could resolve my problem, the collision shape would be a box that is resized whenever the mouse moves(or only when the user releases the mouse). I have seen there is a demo (in the sandbox) and I have copied, pasted and formatted its picking step. However, I have a problem with it (of course), but I think it comes from the initialization of the NewtonCollision shape.
Edi:t or should I use NewtonWorldForEachBodyInAABBDo? But then how can I calculate p0 and p1 (the AABB extent)?
ReEdit: Or is there a function that allows us to use this function NewtonWorldForEachBodyInAABBDo not for a aabb but for a frustum? (pyramid form)
Here is my code:
Code:
Picking function:
- Code: Select all
bool Pick(float x1, float y1, float w, float h, float x2, float y2, fst::mat4 proj, fst::mat4 view){
//x1 = click mouse x, x2 = release mouse x
//y1 = click mouse y, y2 = release mouse y
//w = width of screen, h = height of screen, proj = projection matrix, view = camera view matrix
fst::vec3 p0, p1;
float dx = x2-x1, dy = y2-y1;
p0 = fst::unproject(x1, h-y1, w, h, proj, view, 0.f);
p1 = fst::unproject(x1, h-y1, w, h, proj, view, 1.f);
//I also think I have to change this code for the cases where dx < 0 or/and dy < 0 (I am not sure but I think there might be some problems for those cases)
fst::mat4 matrix = fst::translate(fst::mat4(1.f), p0);
float hitParam;
NewtonWorldConvexCastReturnInfo info[16];
NewtonCollision* const shape = NewtonCreateBox(pl::world, fst::abs(dx), fst::abs(dy), 1.f, 0, NULL); //I create the collision shape with x = dx, y = dy
int count = NewtonWorldConvexCast (pl::world, &matrix[0].x, &p1.x, shape, &hitParam, NULL, NULL, &info[0], 4, 0);
for (int i = 0; i < count; ++i) { // I copy the result to the vector containing the picked bodies
picked.push_back((NewtonBody*)info[i].m_hitBody);
}
asnd here is the code for unproject:
- Code: Select all
inline fst::vec3 unproject(float x, float y, float w, float h, fst::mat4 proj, fst::mat4 mv, float depth){
fst::vec4 r = fst::vec4((x/w)*2.f-1.f,
(y/h)*2.f-1.f,
depth*2.f-1.f,
1.f);
fst::vec4 rw = fst::inverse(proj * mv) * r;
return fst::vec3(rw) / rw.w;
}
Possible Error Zones (My opinion)
- Code: Select all
NewtonCollision* const shape = NewtonCreateBox(pl::world, fst::abs(dx), fst::abs(dy), 1.f, 0, NULL); //I create the collision shape with x = dx, y = dy
- Code: Select all
p0 = fst::unproject(x1, h-y1, w, h, proj, view, 0.f);
p1 = fst::unproject(x1, h-y1, w, h, proj, view, 1.f);
All those snippets are found in the Picking function I provided you with
Question:
Do you know how I can initialize the NewtonCollision with changing x and y values?
Thank you for your answers

If you need more information, please ask me (I can also post short videos if you want)