A very common set of functions that are often used by GUI creators are:
These functions are used to either take a 3D point and convert it to 2D or vice versa (with rays). I think most people can agree that these functions are super useful, but not many people know how they actually work under the hood! As a matter of fact these functions weren't added until close to mid 2015 so what did people do before them?
This tutorial is going to break down what's going on in these functions and how you could (if you were so inclined) write your own function to convert between the 3D world and your screen.
Learning by doing
Now we could start trying to write our function cold turkey, but that's a major waste of time when we have a perfectly good built in function that will help us test some of our hypothesis about how the function works.

Does the camera's CFrame have something to do with it?
 I think it's pretty fair to assume yes on this given just the intuitive thought process that the camera's CFrame represents the position and direction the camera is looking. How could you possibly tell if a 3D position were visible if you don't even have any information about where you're looking from.
 If we really wanted to test this we could run one of the WorldToScreen/Viewport methods while looking at a 3D point. Then only change the camera CFrame to be looking away from it and then run it again to see this has an effect on the method's output (it does!)

Does the camera's field of view have something to do with it?
 Once again we can pretty easily prove that by changing only the FOV value and checking for change in the method's output. If we do this it's very clear that yes, it will have an effect in our function.
 We can find out exactly what the FOV is representing by playing around with the Screen/ViewportToRay functions.
local camera = game.Workspace.CurrentCamera;
local vps = camera.ViewportSize;
 just a module to quickly draw stuff in 3D
 https://www.roblox.com/library/947988936/drawmodule
local draw = require(game.Workspace.draw);
for x = 0, 1 do
for y = 0, 1 do
local r = camera:ViewportPointToRay(vps.x*x, vps.y*y, 0);
draw.ray(r.Origin, r.Direction*1000, game.Workspace);
end;
end;
You'll note that the code above draws a square frustum that represents the area our camera can actually see.
Camera frustum with 70 FOV
We can also note that when we change the FOV in our camera the "spread" of the frustum changes as well.
Camera frustum with 100 FOV (viewed with a 70 FOV)
So you might be wondering how exactly FOV angle comes into play. The FOV angle is simply the vertical angle of our frustum.
Side view of the camera's view frustum
We can easily test this with the following code:
local mtop = camera:ViewportPointToRay(vps.x/2, 0);
local mbottom = camera:ViewportPointToRay(vps.x/2, vps.y);
print(math.deg(math.acos(mtop.Direction:Dot(mbottom.Direction))));
A method to the madness
Now that we know a little bit about what values the methods actually need to work we can start to figure out how our function is going to work.
The first thing we need to do to start is go back to viewing our frustum.
From the above picture we can see a near plane and a far plane. We will define the far plane as the plane that the 3D point we would like to convert to our screen lies on. The near plane is something we will be using more for converting from 2D to a ray, but it's worth mentioning that if we were to move the near plane all the way to the camera's lens then it would represent the "pinhole" that our 3D images are being projected onto. The problem with that of course is that the area of the near plane becomes zero when it's at the camera's lens so we can't actually project onto that.
For now the question we will be asking ourselves is if I wanted to scale down something on the far plane onto another surface with the same aspect ratio how would I do it? Although we won't actually be scaling onto the near plane (for the reasons stated above) it may be helpful to think about it that way.
One way we could do this is, is by finding the percentage x and y that our 3D point (p) lies on far plane from the top left corner. So say for example p was exactly in the center of the far plane then it would be x = 0.5 (50%) and y = 0.5 (50%). This makes it very easy to convert to a different plane as long as we know it's dimensions.
The next question is how do we actually find those percentages? The key is to use the FOV angle and information about the point to find the dimensions of the far plane and convert.
Using SOHCAHTOA makes finding the height of the plane relatively easy.
However, in order to find the width we need to use the aspect ratio to convert. The math behind that is very easy to show algebraically. Say we want some scaling ratio r that tells us the what our x should be given our y. We can simply rearrange that to solve for r:
x = r*y
r = x/y
This might seem like a problem since we need a combination of x and y that gives us the correct r value. The good news is that camera.ViewportSize
allows us to do just that.
So if we use the information above we can get the height and width of the far plane (divided by two) and then use that to find the scale of our 3D point on the far plane:
So our code for finding the far plane dimensions and then finding the percentage x and y would be:
local function pointToViewport(camera, p)
local vps = camera.ViewportSize;
local lp = camera.CFrame:pointToObjectSpace(p);  make point relative to camera so easier to work with
local r = vps.x/vps.y;  aspect ratio
local y = lp.z*math.tan(math.rad(camera.FieldOfView/2));  calc height/2
local x = r*y;  calc width/2
local corner = Vector3.new(x, y, lp.z);  find the top left corner of the far plane
local relative = lp  corner;  get the 3d point relative to the corner
local sx = relative.x / (x*2);  find the x percentage
local sy = relative.y / (y*2);  find the y percentage
local depth = lp.z;  how far away is the far plane from the camera?
local onscreen = depth > 0 and sx >=0 and sx <= 1 and sy >=0 and sy <= 1;
return sx, sy, depth, onscreen;
end;
For those of you familiar with UDim2 you will know that sx and sy can be used immediately (relative to the viewport, not taking top bar into account yet) as the scale values. However if we wanted to convert them into pixels we would just multiply by the respective values in camera.ViewportSize
.
local frame = game.StarterGui.ScreenGui.Frame;
game:GetService("RunService").RenderStepped:connect(function(dt)
local sx, sy, depth, onscreen = pointToViewport(game.Workspace.CurrentCamera, game.Workspace.Part.Position);
if (onscreen) then
frame.Position = UDim2.new(sx, 0, sy, 0);
end;
end);
So that pretty much nails down the WorldToViewportPoint method. So what about WorldToScreenPoint?
This is almost exactly the same as WorldToViewportPoint except we have a 32 pixel high top bar on our screen to account for. So here is some the same function above, but now converting with the screen in mind.
local function pointToScreen(camera, p)
local vps = camera.ViewportSize;
local lp = camera.CFrame:pointToObjectSpace(p);
local r = vps.x/vps.y;
local y = lp.z*math.tan(math.rad(camera.FieldOfView/2));
local x = r*y;
local corner = Vector3.new(x, y, lp.z);
local relative = lp  corner;
local sx = relative.x / (x*2);
local sy = relative.y / (y*2);
 up until now, exactly the same as before.
 convert to viewport pixels
local px = sx * vps.x;
local py = sy * vps.y;
 subtract top bar height from viewport pixels
py = py  32;
 convert back to scale if you so desire
sx = px/vps.x;
sy = py/(vps.y  32);
local depth = lp.z;
local onscreen = depth > 0 and sx >=0 and sx <= 1 and sy >=0 and sy <= 1;
return sx, sy, depth, onscreen;
end;
Putting it in reverse
Okay so we got the 3D to 2D down nicely, but what about the 2D to 3D?
Well this is where the near plane comes into play. If we convert a 2D point on the screen onto the near plane that takes care of the ray.Origin value, and if we just draw a line between our camera and that origin value we get the ray.Direction value. Too easy!
So just working backwards from our previous code, assuming we provide a depth value (or provide a default):
local function viewportToRay(camera, sx, sy, depth)
local vps = camera.ViewportSize;
local depth = depth and math.max(depth, 0.1) or 0.1;
local r = vps.x/vps.y;
local y = depth*math.tan(math.rad(camera.FieldOfView/2));
local x = r*y;
local corner = Vector3.new(x, y, depth);
local lp = corner + Vector3.new(sx*x*2, sy*y*2, 0);
local p = camera.CFrame:pointToWorldSpace(lp);
local direction = (p  camera.CFrame.p).unit;
return Ray.new(p, direction);
end;
Similarly we can do the slight 32 pixel adjustment to do this calculation for a screen point.
local function screenToRay(camera, sx, sy, depth)
local vps = camera.ViewportSize;
local depth = depth and math.max(depth, 0.1) or 0.1;
local py = (vps.y32)*sy + 32;
sy = py/vps.y;
local r = vps.x/(vps.y32);
local y = depth*math.tan(math.rad(camera.FieldOfView/2));
local x = r*y;
local corner = Vector3.new(x, y, depth);
local lp = corner + Vector3.new(sx*x*2, sy*y*2, 0);
local p = camera.CFrame:pointToWorldSpace(lp);
local direction = (p  camera.CFrame.p).unit;
return Ray.new(p, direction);
end;
There you have it the functions that help you intermingle between 2D and 3D!